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RESEARCH REPORT 

New Perspectives on the Validity of the GRE® General Test 
for Predicting Graduate School Grades 

David M. Klieger, Frederick A. Cline, Steven L. Holtzman, Jennifer L. Minsky, & Florian Lorenz 

Educational Testing Service, Princeton, NJ 


Given the serious consequences of making ill-fated admissions and funding decisions for applicants to graduate and professional school, 
it is important to rely on sound evidence to optimize such judgments. Previous meta-analytic research has demonstrated the gener- 
alizable validity of the GRE ® General Test for predicting academic achievement. That research does not address predictive validity 
for specific populations and situations or the predictive validity of the GRE Analytical Writing section introduced in October 2002. 
Furthermore, much of the past GRE predictive validity research is primarily based on approaches that are correlational and univariate 
only. Stakeholders familiar with GRE predictive validity mainly in the form of zero-order correlation coefficients might automatically 
interpret the usefulness of the GRE solely through the prism of Cohens (1988) guidelines for judging effect sizes and without regard 
to the larger context. However, by using innovative and multivariate approaches to conceptualize and measure GRE predictive valid¬ 
ity within the larger context, our investigation reveals the substantial value of the GRE General Test, including its Analytical Writing 
section, for predicting graduate school grades. 

Keywords Predictive validity; utility; GRE; writing; graduate school; graduate school grades; dominance analysis 
doi: 10.1002/ets2.12026 


Graduate and professional education is a limited and precious resource. For a relatively small number of available open¬ 
ings, competitive graduate programs often receive a large number of applications from individuals with impressive cre¬ 
dentials but are able to admit only a small percentage of them. Given the high-level knowledge and skill of those who 
are admitted and then enroll, many graduate and professional students eventually go on to perform some of the most 
consequential work in society. Among their many undertakings, they develop cancer therapies, pursue economic reform, 
and manage geopolitical crises. Furthermore, teaching and training graduate and professional students can be extremely 
labor- and resource-intensive. Figures for actual total costs to programs are difficult to obtain, but the cost for a private 
organization to support a graduate fellow for 3 years comes to $125,000 ($41,667 per annum), excluding administrative 
fees (Wendler et al., 2010). Graduate and professional students and their families sometimes incur enormous financial 
and emotional costs so that students can pursue their education. In 2007-2008, the annual cost to master’s degree stu¬ 
dents ranged from an average of $28,375 to $38,665, and the cost to doctoral students ranged from an average of $32,966 
to $46,029 (Wendler et al., 2010). 1 Often it is the larger society, including taxpayers, who subsidize their training. To the 
extent that private industry finances graduate and professional school students, it shifts resources that might be invested 
toward other worthy causes and might pass costs onto consumers. Considering the stakes, admitting and providing 
resources to a student who will succeed in graduate or professional school should be a top priority. Although some errors 
in admissions and funding decisions might be unavoidable, educational institutions can minimize them by using sound 
assessment tools. The GRE ® General Test is one such tool. Empirical research shows that, with the exception of undergrad¬ 
uate grade point average (UGPA), the GRE General Test predicts higher educational outcomes much more strongly than 
other types of information available for making admissions and funding determinations (e.g., personal statements and let¬ 
ters of recommendation; compare Kuncel, Hezlett, & Ones, 2001, and Kuncel, Wee, Serafin, & Hezlett, 2010, to Borneman, 
Cooper, Klieger, & Kuncel, 2007; Murphy, Klieger, Borneman, & Kuncel, 2009; and Vannelli, Kuncel, & Ones, 2007). 

The main purpose of this research is to use various statistical methods to gain a greater perspective on the predic¬ 
tive validity of the GRE, particularly a better understanding of the practical significance of the zero-order correlation 
metric. The substantial majority of the results in large-scale validity studies of the GRE express predictive validity as zero- 
order correlation coefficients (e.g., Kuncel, Hezlett, et al., 2001; Kuncel et al., 2010; Powers, 2004). Cohen (1988) famously 
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established a set of guidelines to assess the magnitude of a zero-order correlation, where a value of 0.10 is small, 0.30 is 
moderate, and 0.50 is large. However, many scholars explicitly or implicitly question the meaningfulness and usefulness 
of this framework, especially when there is a larger context to consider (see Ellis, 2010; Meyer et al., 2001). Furthermore, 
across program areas, admissions committees usually consider several pieces of information when making admissions 
decisions (Walpole, Burton, Kanyi, & Jackenthal, 2002). A stakeholder might ask if the GRE sections (Verbal Reasoning, 
Quantitative Reasoning, and Analytical Writing) provide any unique information (i.e., over and above what other predic¬ 
tors such as UGPA convey) that allows an admissions committee to better predict who will succeed in graduate school. 
Prior large-scale studies do provide some multivariate information, such as multiple correlations for GRE sections and 
UGPA, and permit one to assess how much variance the GRE explains over and above UGPA. However, this multivariate 
information is not typically broken out by separate domains of study. Additionally, it communicates validity for enrolled 
students only rather than for the applicant pool. We wanted to explore variability in multivariate validity across differ¬ 
ent types of graduate programs for optimally weighted GRE sections combined (as in a regression approach) and for 
incremental validity of the GRE sections over and above the variance in the graduate grade point average (GGPA) out¬ 
come criterion that UGPA explains. Also, we sought to express multivariate validity for the applicant pool and not for 
just enrolled students. Since raw data were available, this study avoided challenges associated with using meta-analytically 
derived matrices of intercorrelations to derive multiple correlations (see discussion in Kuncel, Hezlett, et al., 2001, pp. 
173 -174). In addition, we were able to look at predictive validity (zero-order validity coefficients and multivariate valid¬ 
ity metrics) for the applicant pool by adjusting for range variation via multivariate methods (as did Burton & Wang, 2005, 
and Powers, 2004, for zero-order coefficients). 

A secondary reason for this investigation is to explore the GRE’s predictive validity for a sample from a state university 
system. The authors of GRE validity generalization research for graduate school admissions acknowledge that “there are 
variations in the generally high levels of predictive validity, leaving open questions about the validity of the GRE for 
specific populations and situations” (Kuncel et al., 2010, p. 341). Our aggregation and grouping decisions were based 
on the desire to convey validity information for what we believed were meaningful groups and a concurrent desire to 
maintain reasonable sample sizes (Ns and ks) to minimize sampling error. We did not focus on the genemlizability of 
the validity coefficients in the sense that statistical methods were not used to determine, within the subgroups for which we 
reported validity information (i.e., degree levels and program areas), whether or not there were subpopulations with diverse 
validity coefficients that should be separately reported. As described below, the data are limited to a recent 5-year period for 
public 4-year universities within a single U.S. state. Previous research (Kuncel, Hezlett, et al., 2001) has already empirically 
established the generalizable validity of the GRE examinations. Examination of demographically based differences in 
prediction (usually based on race/ethnicity and/or gender) takes the form of regression-based predictive bias studies. 
With the exception of findings that the GRE examinations over-predict the graduate school performance of some racial 
and ethnic minority populations, studies have not shown statistically or practically significant differences in prediction 
across demographic groups (see Kuncel & Klieger, 2012). 

Methods 

Description of the Database 

The State University System (SUS) of Florida Board of Governors provided much of the data used in this study, including 
applicants’ UGPAs; whether or not applicants to graduate school were admitted and enrolled; and academic records of 
enrolled students, including course names and grades, area of study (i.e., major), and graduation status. SUS data covered 
the academic years 2003-2004 through 2007-2008 and included information for any student who had either applied or 
who was enrolled at any point during those years. SUS data included applicant and enrolled student records from 10 public 
universities with doctoral and/or master’s level graduate programs. Based on the Carnegie classification system, eight of 
the universities are considered doctoral-granting: Of these eight institutions, four of them have very high research activity 
(RU/VH), two have high research activity (RU/H), and two are doctoral/research universities (DRU). The other two insti¬ 
tutions are classified as master’s colleges and universities with larger programs (master’s/L). SUS data initially included 
approximately 400,000 student records (many of which lacked necessary information and were subsequently removed). 

Available GRE data were linked to data that SUS provided. This information included GRE scores for up to three 
GRE General Test administrations and data from the GRE Background Information Questionnaire. GRE data covered 
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all test takers who sent a score report to one of the 10 SUS institutions or who listed one of those 10 institutions as their 
undergraduate school. Test dates ranged from October 2002 (when the GRE Analytical Writing section was added to the 
GRE General Test) through the end of 2007. 2 The GRE data initially contained 240,000 records. 

The resulting matched files included 134,000 records after substantial data cleaning. The matching process employed 
algorithms that took into account missing, incomplete, or mildly discrepant data. Quality control procedures for the 
matching program indicated very high probabilities that data were accurately linked. About 78,000 matched records con¬ 
tained only admission information, and about 56,000 matched records contained course-level data for at least one term. 
We further restricted the dataset of enrolled students to those who had completed at least six graduate school courses 
where a grade was assigned so that we could assess validity for a stable criterion. Certain multivariate methods that 
adjusted effect sizes for range variation (described below) required UGPA information. In order to facilitate comparability 
of various metrics used to measure predictive validity, we wanted all analyses within a degree level (master’s and doctor¬ 
ate) to use the same samples to the extent possible. Therefore, records that lacked UGPA information were removed. 
In addition, records between master’s-seeking and doctorate-seeking students were split for reasons described below. 
Ultimately, data for 25,356 students — 21,127 master’s-seeking students and 4,229 doctorate-seeking students—were ana¬ 
lyzed. Classification of students as master’s seeking or doctorate seeking was based on information that the SUS pro¬ 
vided. Master’s seekers were students who the SUS indicated were seeking only a terminal master’s degree in their SUS 
programs, and doctorate seekers were students who the SUS indicated were ultimately seeking a doctoral degree in 
their SUS programs (either with or without a requirement that a master’s degree be obtained en route in their doctoral 
program). 

Description of the Sample 

Demographics 

Although sample sizes vary slightly across demographic variables, demographic information was available for a relatively 
common set of approximately 20,000 master’s level students and a relatively common set of approximately 4,000 doctoral 
level students. We believe that differences in Ns across these variables are due primarily to a greater ability and willingness 
of GRE examinees to disclose certain demographic information more than other demographic information. Tables 1-8 
provide demographic information for the overall master’s level and doctoral level samples, as well as for specific program 
areas for each of these degree levels. These program areas are based on two-digit Classification of Instructional Programs 
(CIP) codes (National Center for Education Statistics [NCES], 2012). Title IV of the Higher Education Act of 1965 (as 
amended, 2003) requires all postsecondary institutions in the United States that participate in or apply to participate in 
federal student financial aid programs (more than 7,500 institutions) to reply to annual surveys that NCES’s Integrated 
Postsecondary Education Data System (IPEDS) issues to monitor trends in postsecondary education (NCES, 2012, 20 
USC 1094, Section 487[a] [17] and 34 CFR 668.14[b] [19]). Developed with the input of a technical review panel consist¬ 
ing of a broad range of federal agencies, state agencies, and postsecondary institutions, the CIP code taxonomy is the 
accepted U.S. federal standard that NCES created to track fields of study (NCES, 2012). For creating comparison groups 
in research, CIP codes provide flexibility that is useful for graduate-only institutions, as well as institutions that issue bac¬ 
calaureate degrees (see discussion in Sykes, 2011). One could combine two-digit CIP code categories into larger STEM 
(science, technology, engineering, and mathematics), humanities, and social science groupings, especially if one possesses 
a rationale for answering questions about predictive validity at those higher levels of analysis. However, as one can observe 
in the tables presented here, there is variability in the values of predictive validity and other metrics among program areas 
defined by two-digit CIP codes. Averaging these metrics further would result in loss of information that might be useful; 
therefore, we opted to report values that reflect this variance. 

We report here descriptive data for gender, race/ethnicity, age, citizenship/residency, parental education, and degree 
objective when registering for the GRE examinations. Although the substantial majority of master’s level students were 
female (66%), a slight majority of doctoral level students were male (51%). The gender composition for specific program 
areas appears in Tables 1 and 2. Racial and ethnic data appear in Tables 3 and 4. The substantial majority of graduate 
students at both the master’s level (73%) and doctorate level (75%) self-reported as being non-Hispanic and White. Sample 
sizes for American Indian students were extremely small. Asian individuals comprised 3.6% of master’s students and 4.3% 
of doctoral students. Ten percent of master’s degree enrollees were Black students, as were 7.6% of doctoral students. 
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Table 2 Gender, Age, and Citizenship Status: Doctorate Seekers 


D. M. Klieger et al. 


New Perspectives for Predicting Graduate School Grades 
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Note. CIP = classification of instructional programs; M = mean. 








Table 3 Race and Ethnicity: Master’s Seekers 


D. M. Klieger etal. 


New Perspectives for Predicting Graduate School Grades 
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Note. CIP = classification of instructional programs. 





D. M. Klieger etal. 


New Perspectives for Predicting Graduate School Grades 


Approximately 10.2% of master’s students self-reported as being of Hispanic origin, and so did 8.5% of doctoral level 
students. The mean age of master’s level students was 27.3 years, while the mean age of doctoral level students was slightly 
older at 28.6 years. Tables 1 and 2 report age information for specific program areas. As described in Tables 1 and 2, 
the vast majority of students were U.S. citizens, with slightly more master’s level students having U.S. citizenship (92.8%) 
than doctoral level students (86.3%). The highest educational levels of students’ parents sometimes serve as a measure of 
students’ socioeconomic status, so the maximum educational attainment level for either of an enrollee’s parents is reported 
here. For example, if an enrollee’s mother had earned a graduate degree and the enrollee’s father had earned no higher 
than a bachelor’s degree, then we counted that enrollee in the graduate degree category. We believe that, in general, the 
higher of the education levels would most accurately reflect the educational knowledge and influence that parents convey 
to an enrollee, which can affect an enrollee’s educational goals, preparation, and attainment. Educational attainment was 
generally higher for one of the parents of doctoral level students than for one of the parents of master’s level students. Of 
master’s-seeking students, 30.7% had at least one parent who had earned a graduate or professional degree, versus 35.2% 
of doctorate-seeking students. 

Tables 5 and 6 summarize these values across specific program areas and for other levels of educational attainment. 
Seventy-five percent of master’s students indicated that they did not plan to seek a doctorate or postdoctorate position, 
while 24% indicated that they did plan to pursue a doctorate or a postdoctorate position. Among doctoral students, 12.3% 
had previously thought that they would pursue a terminal master’s degree only, and only 2.8% thought that they would 
ultimately go on to a postdoctorate position. Tables 7 and 8 show these values for particular program areas and for less 
common educational objectives. 

Scores on the Predictors (GRE and UGPA) and the Criterion (GGPA) 

One can refer to Tables 9-14 for detailed information for specific program areas. GRE Verbal Reasoning scores for 
enrolled master’s students tended to be only slightly greater than the average for applicants to the SUS system (master’s 
mean = 472; overall test-taker mean = 465), but the master’s student mean was substantially larger for those programs for 
which one would expect verbal ability to play a more central role (e.g., mean = 566 for English language and literature). 
Mean GRE Quantitative Reasoning scores were below the general applicant norm (master’s mean = 553; overall test-taker 
mean = 584), but they were especially high for those SUS programs for which one would expect quantitative ability to play 
a more central role (e.g., mean = 727 for mathematics and statistics). The mean GRE Analytical Writing scores for master’s 
students fell at about the overall test-taker average (4.11) but were higher for master’s level students attending presumably 
more writing-intensive programs (e.g., mean = 4.69 for English language and literature). At the doctoral level, mean scores 
of SUS students were notably greater than the means for applicants to the SUS system, as well as master’s level students 
enrolled in the SUS system: 521 for Verbal Reasoning, 642 for Quantitative Reasoning, and 4.44 for Analytical Writing. 
UGPAs for all students were generally high. For master’s level students, the overall mean was 3.37 (slightly higher than a 
B+ average), and for doctoral students the overall mean was slightly higher at 3.54 (between a B+ and an A— average). 
GGPAs for all students were generally very high. For master’s level students, the overall mean was 3.67 (slightly below 
an A— average), and for doctoral students the overall mean was slightly higher at 3.71 (slightly above an A— average). 
Variability in GGPA was relatively low at both the master’s level (SD = 0.36) and doctoral level (SD = 0.31). Assuming a 
normal distribution of GGPA, about 84% of students attained at least a B+ average in graduate school in both master’s and 
doctoral programs. Although this variability itself varies across program areas, it still seems restricted (never exceeding 
a standard deviation of 0.55 for a master’s level program area or 0.49 for a doctoral level program area). Eighty percent 
of master’s students and 83% of doctoral students never received a graduate school grade below B-, and about 50% of 
master’s level and doctoral level students attained a GGPA of at least a 3.8. 3 Overall and for specific program areas, one can 
find intercorrelations among predictor variables (GRE Verbal Reasoning, GRE Quantitative Reasoning, GRE Analytical 
Writing, and UGPA) for applicants in Tables 13 and 14. Intercorrelations among the GRE sections generally are larger 
than they are between GRE sections and UGPA. 

Meta-Analysis and Statistical Artifacts 

We conducted meta-analyses on the data in several ways, partly based on Hunter and Schmidt’s (2004) approach for 
correlation-based validity coefficients and partly based on other methods. 4 As estimations of relationships involving the 
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Overall (all general programs combined) 10 20,991 75.0(15,749) 23.3(4,883) 0.7(137) 0.9 (186) 0.1 (17) 0.1 (19) 
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Table 11 Descriptive Statistics for Graduate Grade Point Average (GGPA) and Undergraduate Grade Point Average (UGPA): Master’s Seekers 
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Table 12 Descriptive Statistics for Graduate Grade Point Average (GGPA) and Undergraduate Grade Point Average (UGPA): Doctorate Seekers 
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Note. CIP = classification of instructional programs; GRE-V = GRE Verbal Reasoning; GRE-Q = GRE Quantitative Reasoning; GRE-AW = GRE Analytical Writing; 
UGPA = undergraduate grade point average 
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measurement of latent characteristics for a population of interest, all statistics are vulnerable to statistical artifacts that 
make their interpretation more difficult. In this study, the statistical artifacts addressed were sampling error, range varia¬ 
tion, and unreliability in the criterion variable (GGPA) only. 

Sampling Error and Aggregation Decisions 

Sampling error occurs when the data available for analysis (e.g., data for just the graduating class of 2010) are for just a 
sample (subset) from a larger population (e.g., all students) about which the researcher wishes to answer a research ques¬ 
tion (e.g., “What is the relationship between admission test scores and future grade point average for all students?”). Due 
to various causes, samples never perfectly represent the larger populations from which they come. Sampling error causes 
data to randomly misrepresent the population about which a researcher wishes to draw conclusions. One addresses this 
issue by increasing sample size to increase the probability that the sample represents the population sufficiently well. In 
this study, sampling error was addressed by aggregating, separately for doctoral versus master’s level programs, all data 
based on two-digit CIP codes within each of the 10 institutions (NCES, 2012). In this particular study, aggregation using 
CIP codes helps us to more authoritatively say what the validity of the GRE sections are for the domains defined by these 
codes. This data aggregation procedure consisted of computing validity metrics (zero-order correlations, multiple correla¬ 
tions, incremental coefficients of determination, usefulness weights, and quartile comparisons) and criterion unreliability 
for each two-digit CIP code for each degree level of the 10 universities. Validity metrics were adjusted for range variation 
(and values necessary to make that adjustment were calculated) for each two-digit CIP code for each degree level of the 10 
universities. This is equivalent to treating each two-digit CIP code at each degree level within each university as if it were 
its own study. After computing each metric (and after making any adjustments), they were averaged within degree level 
and across universities after weighting by sample size. Ultimately, these averages are reported for each two-digit CIP code 
for each degree level (see the appendix). As a result of this aggregation approach, these averages can at least attempt to 
properly account for unique aspects of each degree level within university (e.g., differences in graduate grading stringency 
across universities within degree levels). 

The approach of examining predictor-criterion relationships for master’s programs separately from doctoral programs 
is consistent with methodology that Kuncel et al. (2010) and Burton and Wang (2005) pursued. This method was followed 
in the belief that between these two levels of degree programs there might be practically significant differences in (a) the 
knowledge, skills, abilities, and other relevant characteristics (KSAOs) of their respective populations, as well as (b) the 
determinants of success (e.g., ability, motivation, etc.). Kuncel et al. (2010) found some differences in both adjusted and 
unadjusted validity coefficients between master’s and doctoral programs. Even though many of these differences were 
relatively small, the current analyses go further than Kuncel et al. (2010), by looking at validity metrics beyond zero-order 
coefficients and subpopulations at the program area level. 

There were 28 master’s level program areas and 26 doctoral level program areas for which data were available and defin¬ 
able by two-digit CIP codes. For each degree level, some program areas had small sample sizes, as defined by the small 
number of students ( N ) and the small number of universities ( k ) that contributed data. Consequently, our discussion 
of results (presented here) is limited to a subset of these program areas. This subset, consisting of eight program areas, 
includes education; engineering; English language and literature/letters; biological and biomedical sciences; mathemat¬ 
ics and statistics; psychology; health professions and clinical sciences; and business, management, and marketing. For 
business, management, and marketing, we excluded results for doctoral programs, because there were only 16 enrolled 
doctoral students in total across four universities. These eight program areas were chosen based on their large sample 
sizes and their ability to represent a wide range of domains, including STEM, social science, humanities, and applied 
professional fields. In Tables 15-22, a prominent separation was placed between applied professional fields, namely 
health professions and clinical sciences as well as business, management, and marketing, from the other fields because 
the history of using the GRE for admissions in applied professional fields is much shorter than that for admissions in 
other areas. Only students in the applied professional fields who were seeking either a master’s or doctoral degree were 
included in the data (e.g., anyone seeking an M.D. degree was removed), because it was assumed that the presence of 
a GRE score for a student seeking any other degree was coincidental and did not indicate that the GRE test was being 
used to make an admissions decision for that student for the program for which the student’s graduate school grades 
were available. 
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Table 15 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for Eight Program Areas: Master’s Seekers 
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Table 19 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for Seven Program Areas: Doctorate Seekers 
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Table 20 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Doctorate Seekers 


D. M. Klieger et al. 
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R 2 = proportion of variability in the criterion (graduate GPA) predicted; A R 2 over UGPA = amount of variability of the criterion predicted by GRE sections over and above what UGPA 
alone predicts; the subscript +UGPA indicates that the UGPA variable was included when calculating values; %C X j = usefulness (general dominance) weight converted into the percentage 
of the R 2 attributable to a variable. 
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Range Variation 

Range variation describes the systematic effect on the magnitude of validity coefficients that results from the researcher 
possessing data for only enrolled students instead of the entire applicant pool. Generally, outcomes data (e.g., future GPA) 
are available for only those who have actually been accepted and then enrolled, rather than for the entire applicant pool. 
However, the researcher usually wants to know validity for the entire applicant pool. In other words, the researcher usually 
wants to know the answer to the question, “If every applicant had been accepted and enrolled— and, thus, I had outcomes 
data for all applicants—what would the validity metric (e.g., validity coefficient) equal?” In most admissions contexts, 
range variation takes the form of range restriction in that the predictor scores for the applicant pool usually are more 
varied than the predictor scores for the enrolled pool of students. Institutions generally select applicants with higher 
predictor scores, which results in a restriction (a narrowing in the range) of scores as one shifts focus from the applicant 
pool to the enrolled pool. Adjustments for range restriction will increase the magnitude of correlation coefficients. In 
some cases, range variation could take the form of range enhancement, in that the predictor scores for the applicant pool 
would be less varied than the predictor scores for the enrolled pool of students. It is possible that mainly those applicants 
with low or high predictor scores would be the ones to enroll in a program. This phenomenon would be more likely for 
less selective and less prestigious programs and for programs with an applicant pool whose predictor scores were not very 
variable. Adjustments for range enhancement would decrease the magnitude of correlation coefficients. 

Range variation was addressed both univariately, using methods recommended by Hunter and Schmidt (2004), and 
multivariately, using methods recommended by Lawley (1943). The univariate adjustment is based on the following 
formula: 

U*P2 

Ul-l)p\ + l 

where p x = the validity coefficient adjusted for range variation, p 2 = the observed (unadjusted) validity coefficient, and 
U = — (where er , = the standard deviation for the applicant pool and = the standard deviation for enrolled students; 
Hunter & Schmidt, 2004, p. 107). This formula assumes that a graduate program selected the applicant directly on the 
predictor (rather than on another independent variable that is correlated with that predictor). Each validity coefficient 
for each two-digit CIP code for each university was adjusted using this equation before the adjusted coefficients were 
sample-size weighted and averaged across universities for each two-digit CIP code within degree level. 

The multivariate procedures were based on using matrix operations to solve for the elements of the following: 

^(r)a ^(r)fe 
V T y ’ 

\r)b hr)c_ 

where dimensionally is (p X p), is (p X (« —p)), and S (r)c is ((« -p) X (n —p)) (see Lawley, 1943). In some of the 

literature on multivariate adjustment, £( r ) is alternatively notated as v (e.g., Ree, Carretta, Earles, & Albert, 1994). E( r ) a 
(sometimes notated as v pp ) = the variance-covariance matrix for predictor variables based on the entire applicant pool; 

(sometimes notated as v pn _ p or v n _ pp ) = the covariances between predictors and the criterion based on the entire 
applicant pool; and (sometimes notated as v n _ p n _ p ) = the variance of the criterion based on the entire applicant pool. 
Only X( r ) a is knowable, because values for predictors, but not the criterion, are available for those applicants who do not 
enroll. Nevertheless, one can estimate the remainder of the elements of with equations that incorporate I,^ a and data 
for the enrolled students (see Ree et al., 1994). We solved Z. (rj for each two-digit CIP code for each university within each 
degree level before the adjusted coefficients were sample-size weighted and averaged across universities and within degree 
level for each two-digit CIP code. These multivariate methods can be time-consuming to execute, might be less familiar to 
consumers of validity data, and require assumptions that the univariate methods do not (i.e., that the relationship between 
independent variables is linear). Furthermore, past validity studies (e.g., Kuncel, Hezlett, et al., 2001; Kuncel et al., 2010) 
to which one may want to compare validity coefficients used univariate adjustment procedures. Nevertheless, only the 
multivariate methods account for the overall impact of range variation on multiple predictors that are intercorrelated and 
collectively used in selection decisions. This is especially advantageous given that graduate admissions committees may 
use GRE sections and UGPA in a compensatory manner such that a high score on one predictor may offset a low score on 
another. Ruscio (1998, p. 569) observed that compensatory selection often “stacks the deck squarely against the predictive 
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validity of the GRE,” because it depresses the magnitude of validity coefficients. Like Powers (2004) and Burton and Wang 
(2005), we employ multivariate procedures to avoid this risk. 

Unreliability (Measurement Error) 

Unreliability in the criterion systematically attenuates the magnitude of effect sizes (Hunter & Schmidt, 2004; Spearman, 
1904). The inconsistency with which a criterion (or independent variable) assesses students places a ceiling on how accu¬ 
rately an independent variable may predict that criterion. One could adjust (increase) the magnitude of the attenuated 
correlation by dividing it by the square root of reliability, or, in the case of an imperfectly reliable predictor and criterion, 
the product of the square roots of their reliabilities (Hunter & Schmidt, 2004; Spearman, 1904). In an operational context, 
a predictor is accountable for attenuation that its own unreliability causes. Arguably, it is not accountable for attenuation 
that the criterion causes even if that unreliability is an ingrained part of the operational context. On the other hand, mak¬ 
ing adjustments to a validity coefficient based on criterion unreliability does not reflect operational reality. The use or 
non-use of the predictor will not change a criterion that a graduate program uses, no matter how unreliable the criterion 
might be. As a compromise among competing considerations, we noted for each validity coefficient what the reliability of 
the criterion was and indicated the percentage decline in magnitude of a perfectly measured validity coefficient associated 
with that unreliability (1 — Wr Y y, where r YY = criterion reliability). For each two-digit CIP code for each degree level, we 
estimated measurement error in the criterion using Cronbach’s alpha for students who had at least 10 course grades. As 
with adjustments for range variation, we calculated reliability coefficients for each two-digit CIP code for each university 
within each degree level before the adjusted coefficients were sample-size weighted and averaged across universities for 
each two-digit CIP code for each degree level. 

Relative Importance/Usefulness and AR 2 for Enrolled Students and the Applicant Pool 

For communicating the importance of a variable in prediction, the usefulness weight from dominance analysis ( C X j ) is 
superior in several ways to zero-order correlations, regression weights (whether nonstandardized ps or standardized /is), 
incremental variance explained (AR 2 s), and f-statistics (Budescu, 1993; J. W. Johnson, 2000). The usefulness weight is 
“the proportionate contribution each predictor makes to R 2 , considering both its unique contribution and its contribution 
when combined with other variables” (J. W. Johnson, 2000, p. 1). It measures a predictor’s direct effect when considered 
by itself, its total effect conditional on all other predictors, and partial effect conditional on subsets of predictors (Budescu, 
1993). The usefulness weights of a set of predictors always sum to the R 2 for the regression model containing all of those 
predictors. In an analysis of general dominance, one calculates a usefulness weight for each variable by computing its AR 2 
for a full regression model (containing that predictor) versus the full model without the predictor, as well as its Aft 2 s for 
all possible submodel comparisons. Since it is often unclear exactly which variables from the full regression model will 
actually be used in the future for decision making, in general dominance analysis all of these Aft 2 s are then averaged 
together to describe the variable’s average usefulness. Azen and Budescu (2003) explained that this particular averaging 
methodology is an analysis of general dominance, as opposed to an analysis of each variable’s contribution to R 2 across 
all regression models of a certain size (complete dominance analysis) or an analysis of each variable’s average additional 
contribution to R 2 within regression models of a certain size (conditional dominance analysis). Using bootstrap methods, 
they showed that analysis of general dominance leads to very stable and replicable results. Since the usefulness weights 
sum to the R 2 for the full model, each usefulness weight can be thought of as a percentage of that R 2 (Budescu, 1993). 
In the end, one possesses ratio-level metrics of importance that describe how much of the predictable variance of the 
criterion is uniquely attributable to each independent variable on average. 

There are several reasons why alternative metrics of predictor importance are problematic indicators of variable impor¬ 
tance. If predictors are intercorrelated, then a zero-order correlation cannot communicate the unique contribution of an 
independent variable to prediction. A nonstandardized regression weight often cannot be compared to another nonstan¬ 
dardized regression weight, because each is dependent on the unit of measurement underlying its predictor (e.g., inches, 
feet, yards, etc.). For comparability, they would have to be standardized, but even highly intercorrelated, standardized 
predictors that are highly correlated with the criterion may have highly discrepant regression coefficients (J. W. John¬ 
son, 2000). Regression weights can suffer from multicollinearity and/or suppressor effects (Budescu, 1993). J. W. Johnson 
(2000) enumerates other limitations, such as the exaggeration of the standardized beta weight for the predictor most 
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highly correlated with the dependent variable and a corresponding decline in the magnitude of the standardized beta 
weights of the other predictors. The f-statistics are not a viable alternative either, because they fail to account well for the 
intercorrelation of predictors (J. W. Johnson, 2000). 

We did not locate any previous large-scale (or GRE) predictive validity studies that have used usefulness weights. Also, 
not only usefulness weights were calculated for GRE sections and UGPA for enrolled graduate students with GGPAs, but 
calculations using R 2 sfor the applicant pool (i.e., based on multivariate range variation adjustments) were also performed. 
As was first shown by Lawley (1943) and later elaborated upon by others (e.g., Birnbaum, Paulson, & Andrews, 1950), if the 
relationship between the dependent and independent variables is linear and the associated residuals are homoscedastic, 5 
then the unknown parts of £ a , and £ c can be obtained from £( r y 




(1) 


r)c ^(r)b (^(r)a ^(r)a^a^(r)rt) ^(r)b 

Given the unrestricted covariance matrix (the matrix representing the entire applicant pool) obtained in this manner, 
one can compute the correlation matrix p as follows: p = D~ 1 ZD - where D = (£°f) 5 , that is, the square root of the 
diagonal matrix resulting from a Eladamard (i.e., element by element) product of £ and the identity matrix. The correlation 
matrix allows the computation of the coefficient of multiple correlation through the relationship R 2 = a 1 p~^ a, where p xx 
is the matrix of correlations among the independent variables, and £ is the vector of correlations between the dependent 
and independent variables. These procedures also allow one to calculate incremental variance explained (AR 2 s) by the 
GRE and GRE sections over UGPA for predicting GGPA for the applicant pool in addition to traditional AR 2 s based on 
observed data only. 


Quartile Comparisons 

Bridgeman, Burton, and Cline (2009) suggested a valuable comparison of outcomes for top versus bottom GRE score 
quartiles of enrolled students to illustrate predictive validity. Limitations of this approach include omitting part of the score 
distribution, communicating validity for enrolled students rather than the applicant pool, and conveying only univariate 
information when decision makers might simultaneously use more than one predictor. However, comparisons for top 
versus bottom GRE score quartiles more directly communicate the value of the GRE than do correlations, incremental 
coefficients of determination, and usefulness weights. This straightforward approach avoids statistical adjustments and 
assumptions that underlie alternative, more time-consuming methods and that may not fully hold (e.g., homoscedasticity). 
We also wanted to compare these alternative methods to a more straightforward approach to see the extent to which they 
are consistent in conveying validity. Therefore, we conducted GRE score quartile comparisons where the outcomes were 
(a) the probability of receiving a C+ or lower in a graduate course and (b) the probability of obtaining a graduate GPA of 
3.8 or higher. As our review of the Internet revealed, some graduate schools and departments warn, place on probation, 
or dismiss a student receiving a C+ or lower in a graduate course, and, even with grade inflation, a graduate GPA of 3.8 
can represent an important milestone of success. 


Results and Discussion 

Four types of validity analyses were conducted and reported in separate tables for each degree level: zero-order validity 
coefficient analyses with and without adjustments for range variation (Tables 15 and 19), multivariate validity and 
incremental validity analyses without adjustments for range variation (Tables 16 and 20), multivariate validity and 
incremental validity analyses with adjustments for range variation (Tables 17 and 21), and GRE quartile comparisons 
(Tables 18 and 22). 


Zero-Order Validity 

For the zero-order validity coefficient analyses, predictive validity was first examined based on zero-order validity coeffi¬ 
cients (r) for each GRE section (reported in Tables 15 and 19). For reasons given previously, we did not adjust observed 
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validity coefficients ( r obs ) for unreliability but do report the reliabilities (Cronbach’s alphas) and the impact of unreliabil¬ 
ity on true validity (the coefficient when there are no statistical artifacts attenuating it). The reliabilities ranged from 0.64 
to 0.76 for master’s programs and 0.57 to 0.71 for doctoral programs (see Tables 15 and 19, respectively). These values 
were fairly narrow in range and generally smaller than criterion reliability values for grades used by Kuncel, Hezlett, et al. 
(2001) and Kuncel et al. (2010) for graduate programs (0.84,0.84, and 0.80) and by Powers (2004) for veterinary programs 
(0.74 to 0.98, median = 0.92) to adjust the observed validity coefficients that they had obtained. Kuncel, Hezlett, et al.’s 
adjustments to GRE validity coefficients were based on the reliability of undergraduate grades, and Powers’s estimates for 
the GRE were based on veterinary school data. If Kuncel, Hezlett, et al. (2001) would have used our estimates to adjust 
their validity coefficients, these coefficients mathematically would have resulted in even larger adjusted values, because 
our reliability estimates were smaller in magnitude than theirs (see Hunter & Schmidt, 2004; Spearman, 1904). For the 
current dataset, attenuation in the validity coefficient due to unreliability ranged from 13% to 20% for validities in master’s 
programs and 16% to 24% for validities in doctoral programs (see Tables 15 and 19, respectively). The reductions in the 
magnitude of the validity coefficients indicate that the unreliability of the criterion is substantially penalizing the zero- 
order validity coefficients for the GRE examination, even though the GRE examination is not at fault. Nevertheless, it is 
expected that GGPA will continue to be the leading measure of academic achievement in graduate education; therefore, 
we did not adjust for measurement error in the criterion and instead accepted the consequence of criterion unreliability. 

In order to account for the effect of range variation on the size of observed validity coefficients ( r obs ), we separately 
report validity coefficients that were adjusted using univariate and multivariate methods. The r urv values reflect the 
observed validity coefficients adjusted using univariate methods described previously (see Tables 15 and 19). The 
U^-ratio (the standard deviation for the applicant pool divided by the standard deviation for enrolled students) is 
reported for each coefficient (Hunter & Schmidt, 2004, p. 107) to reflect the amount of univariate range restriction (or 
enhancement) on the observed validity coefficient. In other words, the U RR -ratios indicate how well an observed validity 
coefficient represents predictive validity for the applicant pool, rather than solely for the enrolled students. Values less 
than 1 indicate range enhancement, and greater values indicate range restriction. The closer the value is to 1, the better 
an observed validity coefficient represents predictive validity for the applicant pool. A large U^-ratio might indicate a 
program area’s particular reliance on a GRE section in admissions. For all program areas for all sections at the master’s 
and doctoral levels, the values are larger than 1, indicating widespread range restriction. For both master’s and doctoral 
programs, the greatest univariate range restriction was for mathematics and statistics, as well as for psychology for the 
GRE Quantitative Reasoning section (U RR = 1.24-1.48; see Tables 15 and 19). For those program areas, it is possible 
that the GRE Quantitative Reasoning section was used substantially to filter out applicants. Given the importance 
of quantitative ability to classroom success in mathematics and statistics graduate programs, as well as a number of 
psychology programs that heavily rely on quantitative reasoning skills, this finding is not surprising. There are several 
cases where explanations for considerable range restriction are less clear (e.g., U RR = 1.26 for GRE Verbal Reasoning in 
engineering master’s programs; U RR = 1.27 for GRE Analytical Writing in mathematics and statistics doctoral programs; 
see Tables 15 and 19). It is possible that engineering master’s programs heavily use the GRE Verbal Reasoning section and 
that mathematics and statistics doctoral programs heavily use the GRE Analytical Writing section to distinguish among 
candidates that the programs perceive to be too homogenous in (high) quantitative ability. 

Given the substantial possibility that graduate programs engage in compensatory selection involving the GRE sections 
and UGPA, it seems most appropriate to communicate predictive validity to reflect the multivariate nature of admissions. 
We report the observed coefficients ( r obs ) to show the effect of making adjustments that account for the effects of range 
variation on the size of validity coefficients. Furthermore, we report univariately adjusted values ( r mv ) to (a) show that 
the multivariate procedures, in fact, do make an additional difference by reflecting the combined effect of intercorrelated 
predictors on selection and (b) provide coefficients that are comparable to GRE validity coefficients reported in Kuncel, 
Hezlett, et al. (2001) and Kuncel et al. (2010). Multivariate methods that do not consider UGPA ( r mrv ) are reported along 
with those that do {r mrv+UGPA ), because (a) it is possible that some programs consider the GRE sections but not UGPA 
(i.e., when UGPAs come from an array of universities with varying grading standards), and (b) when making multivariate 
adjustments to the coefficients of GRE sections, it is important to understand the direction and degree of change in the 
coefficients’ magnitudes when adjustment procedures consider additional predictors, such as UGPA. For both master’s 
and doctoral programs, the general trend is for an increase in the size of the GRE validity coefficients as one shifts focus 
from (a) observed correlations ( r obs ) to (b) univariately adjusted ones (r urv ) to (c) multivariately adjusted ones that exclude 
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UGPA from the adjustment procedure ( r mn ) to (d) multivariately adjusted ones that include UGPA in the adjustment 
procedure {r mrv+UGPA ; see Tables 15 and 19). Reversals in this trend occur but are the exception. 

Based on Cohen’s (1988) standards to judge the size of zero-order correlation coefficients, one might conclude that 
the GRE examination does not provide much predictive value. For each GRE section at the master’s level, virtually all 
of the GRE zero-order validities adjusted for multivariate range variation with procedures that consider the impact of 
UGPA are what Cohen classifies as small to medium in magnitude (i.e., in the range of 0.10-0.30). The sizes of zero- 
order correlation coefficients for the GRE Verbal Reasoning and Quantitative Reasoning sections in predicting GGPA 
in master’s level psychology programs are even smaller. For our eight exemplar master’s programs, validity coefficients 
adjusted to account for the effects of multivariate range variation with procedures that consider the impact of UGPA 
can be found in Table 15. (For all master’s level program areas, the adjusted coefficients appear in the appendix.) For 
GRE Verbal Reasoning, overall r mrv+UGPA = 0.17 (N = 21,127, k= 10), and r mrv+UGPA values for specific program areas 
range from 0.09 for psychology (AT = 461, k = 9) to 0.32 for English language and literature/letters (N = 552, k = 9 ). For 
GRE Quantitative Reasoning at the master’s level, overall r mrv+UGPA = 0.15 {N = 21,127, k = 10), and r mrv+UGPA values for 
specific program areas range from —0.01 for psychology (N = 461, k = 9) to 0.21 for mathematics and statistics (N = 230, 
k = 8), as well as health professions and clinical sciences (N = 3,772, k = 10). For GRE Analytical Writing at the master’s 
level, overall r mrv+UGPA = 0.19 ( N = 21,127, k= 10), and r mrv+UGPA values for specific program areas range from 0.15 for 
business, management, and marketing (N = 901, k = 9) to 0.33 for English language and literature/letters ( N = 552, k= 9). 
We note that the Analytical Writing section, introduced in October 2002, is often the strongest predictor of GGPA. 

Adjusted zero-order validity coefficients for doctoral programs are somewhat larger than those for master’s level pro¬ 
grams, but, based on Cohen’s (1988) rules of thumb, would still generally be considered small to moderate in size. For our 
seven exemplar program areas at the doctoral level, validity coefficients adjusted to account for the effects of multivariate 
range variation with procedures that consider the impact of UGPA can be found in Table 19. (For all doctoral level pro¬ 
gram areas, they can be found in the appendix.) For GRE Verbal Reasoning, overall r mrv+UGPA = 0.19 {N = 4,229, k = 10), 
and r mrv+UGPA values for specific program areas range from 0.08 for engineering (N = 670, k= 6) to 0.30 for biological 
and biomedical sciences ( N = 446, k = 6). For GRE Quantitative Reasoning at the doctoral level, overall r mrv+UGPA = 0.24 
(N = 4,229, k= 10), and r mrv+UGPA values for specific program areas range from 0.10 for English language and litera¬ 
ture/letters (N = 137, k = 4) to 0.41 for mathematics and statistics ( N = 122, k = 5). For GRE Analytical Writing at the 
doctoral level, overall r mrv+UGPA = 0.21 (N = 4,229, k = 10), and r mrv+UGPA values for specific program areas range from 
0.10 for engineering ( N = 670, k = 6) to 0.32 for psychology (N = 417, k = 6). As with prediction at the master’s level, the 
Analytical Writing section is often the strongest predictor of GGPA. 

Usefulness (Utility) Based on Zero-Order Validity 

As did Kuncel, Hezlett, et al. (2001), we question the assumption that large zero-order validity coefficients for the GRE 
are necessary to justify the usefulness of the GRE for predicting graduate school outcomes criteria. In general, highly 
important predictive relationships do not necessarily involve large zero-order correlation coefficients. For example, based 
on large-scale studies, the correlation (a) between taking aspirin and reducing the risk of a heart attack is a mere 0.02, (b) 
between the effect of divorce and a child’s well-being is just 0.09, and (c) between the amount of destruction in brain tissue 
and learning impairment in monkeys and humans is only about 0.17 (Meyer et al., 2001). Even a biological relationship 
as seemingly in lockstep as U.S. adult height and weight is approximately 0.44 (Meyer et al., 2001). Given the results of 
these large-scale studies, highly important predictive relationships do not necessarily involve large zero-order correlation 
coefficients. Moreover, the utility literature also casts doubt on the requirement of large validity coefficients to signify 
usefulness. For example, if only half of an organization’s current students (or employees) are satisfactory performers, 
and an organization uses top-down selection to select 80% of applicants based on a selection system with a validity 
coefficient of only 0.05, then more than half of the selected applicants will be successful students (or employees; Taylor & 
Russell, 1939). 

Financially based analysis also puts into question a need for large zero-order validity coefficients to substantiate use of 
an assessment. According to Hunter and Hunter (1984), the gain (in dollars) from utilizing a selection system for 1 year 
can be calculated as follows: 

N * T * r xy * G v * X, 
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where N = the number of those selected; T = the average tenure of someone selected; r xy = the validity coefficient of the 
predictor; <y = the economic value (usually in dollars) of 1 standard deviation of performance on the criterion per year; 
and X = the average performance on the predictor of selectees, in z-score units. 6 Although the validity coefficient itself 
can be no larger than 1, the effect of being multiplied against several values that usually are larger than 1 (some much 
larger) results in a product that is fairly sizeable. 

A financial example of the usefulness to a graduate program of an assessment used to admit full-time Ph.D. students is 
somewhat simplified but, nevertheless, instructive. According to the National Research Council (2010), from 2002 to 2006, 
the average size of an entering class for a Ph.D. program was 5.8 students for the humanities and slightly larger for other 
broad fields of study. Envisioning a doctoral program that admits five doctoral students in a year (N = 5) is reasonable. 
Between the academic years 1992 -1993 through 2003-2004, only 23% of Ph.D. students completed their doctoral degree 
in 5 years (Sowell, Zhang, Redd, & King, 2008), so a timetable of 5 years for degree completion is a conservative one 
(T = 5). The average performance on the predictor of selectees in z-score units ( X ) for even moderately selective programs 
(admitting 30% of applicants) is greater than 1 (e.g., 1.17 for a 30% selection ratio; Hunter & Hunter, 1984). 

The economic value of 1 standard deviation of doctoral student performance in dollars ( a Y ) is particularly difficult 
to determine, as is the broader issue of the economic value that graduate students provide. One can make the debatable 
assumption that the economic value that doctoral students provide reflects, at a minimum, the economic value provided 
to them in return through wages, stipends, waivers, and subsidies. Most doctoral students provide value to programs 
through tuition, fellowships, research assistantships, and/or teaching assistantships. Thus, in practice, <r Y can measure 
several criteria as well as multidimensional criteria. For purposes of this example, assumptions about the criterion are 
necessarily simplified. Based on a 2008-2009 survey that the Chronicle of Higher Education (June, 2008) conducted with 
a geographically diverse group of 111 public and private not-for-profit universities, the lowest average annual stipend for 
graduate assistants was $13,265 (for sociology, based on the reports of 82 institutions), and the highest average stipend was 
$18,270 (for biology, based on the reports of 94 institutions). Ehrenberg, Klaff, Kezsbom, and Nagowski (2002) estimated 
that, for the 2000-2001 academic year, the average salary of a teaching assistant at a public research university without 
labor union representation of graduate students was $11,817. The foregoing dollar estimates of compensation exclude 
the economic value of subsidies for health insurance, as well as tuition and fee waivers. Eighty-four percent of schools 
surveyed by the Chronicle of Higher Education reported subsidizing at least part of graduate students’ health insurance 
costs (June, 2008). In addition, the amount of waivers and subsidies that doctoral students receive is substantial. The 
average annual tuition for a full-time doctoral degree program in 2007-2008 (but measured in 2009-2010 dollars) was 
$11,900 for public institutions and $24,200 for private not-for-profit ones (NCES, 2011, Table A-48-1). In 2007-2008 
(but measured in 2009-2010 dollars), average annual grants, assistantships, and other nonloan aid for full-time doctoral 
students totaled $20,300 ($9,100 in grants alone) at public institutions and $19,300 ($10,800 in grants alone) at private 
not-for-profit institutions (NCES, 2011, Table A-48-1). If one looks at just the subset of full-time doctoral students who 
received financial assistance (92% at public institutions and 94.5% at private not-for-profit ones), annual nonloan aid 
jumps to $30,700 ($14,100 in grants alone) at public institutions and $34,000 ($17,500 in grants alone) at private not-for- 
profit ones (NCES, 2011, Table A-48-2). 

After finding reasonable inputs for calculating the economic value that doctoral students provide, the process for 
calculating a Y becomes fairly mechanical. Based on Schmidt and Hunter’s (1983) empirically derived finding that value of 
output to pay is about 2 to 1, they advise deriving o Y by taking 40% of annual wages (see discussion in Hunter & Hunter, 
1984). If wages are defined more broadly to include the dollar amounts described above for the lower end of stipends 
($13,265) combined with tuition waivers ($11,900 for public universities and $24,200 for private not-for-profit ones), 
then a Y for public universities would equal $10,066 (40% of $25,165) and for private not-for-profit universities $14,986 
(40% of $37,465). Whatever the predictor’s validity coefficient is, in the case of public universities, it is being multiplied by 
$294,430 per year based on modest assumptions (i.e., 5 admittees * 5 years’ tenure * o Y of $10,066 * 1.17 average perfor¬ 
mance on the predictor = $294,430). Even a validity coefficient of 0.05 would result in a gain of $14,721 (i.e., $294,430 * 
0.05 = $14,721). Under this scenario, as long as total cost to the graduate program for using a predictor with a 0.05 validity 
coefficient is below $14,721, then the net gain/loss from using that predictor is positive. Based on the same methodology, 
for private not-for-profit programs, the multiplier for the validity coefficient is $438,340.50 (i.e., 5 admittees * 5 years’ 
tenure * o Y of $14,986 * 1.17 average performance on the predictor = $438,340.50). This results in a gain of $21,917 even 
if the validity coefficient is a mere 0.05 (i.e., $438,340.50 * 0.05 = $21,917). Provided that the cost to the graduate program 
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for using a predictor with a 0.05 validity coefficient is below $21,917, then the net gain/loss from using that predictor 
is positive. 

Although the inputs to calculate utility (including costs) will vary across specific program areas (and programs), one 
should note that the validity coefficients shown in Tables 15 and 19 (and even the values for incremental validity of the 
GRE test and its sections over and above UGPA depicted in Tables 16 and 20) are usually substantially larger than 0.05. 
Such an observation indicates that the financial benefits from using the GRE test (overall and individual sections) to make 
admissions and funding decisions can be considerable. Using the conservative assumptions above for public universities 
together with overall doctoral r mrv+UGPA values for the GRE Verbal Reasoning, GRE Quantitative Reasoning, and GRE 
Analytical Writing sections from Table 19 (0.19, 0.24, and 0.21, respectively), the annual gain (without consideration of 
costs) to an SUS doctoral graduate program would be $55,940 just from using the GRE Verbal Reasoning section, $70,661 
just from using the GRE Quantitative Reasoning section, and $61,828 just from using the GRE Analytical Writing section. 
If these calculations instead had used observed zero-order validity coefficients unadjusted for range variation ( r obs values in 
Table 19 of 0.13,0.15, and 0.16, respectively), an SUS doctoral program’s annual gain (without consideration of costs) still 
would have been $38,282 just from using the GRE Verbal Reasoning section, $44,175 just from using the GRE Quantitative 
Reasoning section, and $47,123 just from using the GRE Analytical Writing section. 

Multivariate Metrics for Validity 

Validity metrics beyond zero-order correlations substantiate that the GRE test provides important predictive value. Mul¬ 
tiple correlations (Rs) for the GRE Verbal Reasoning, GRE Quantitative Reasoning, and GRE Analytical Writing sections, 
adjusted for multivariate range variation, illustrate that combining the sections into a regression model leads to more 
moderate effect sizes (i.e., often 0.30 and higher) as measured by Cohens (1988) standards. 7 Unlike traditional regression 
approaches, which reflect validity for enrolled students only (and for which results are reported in Tables 16 and 20), these 
adjusted values represent predictive validity for the entire applicant pool. For programs at the master’s level, the overall 
R (R 2 ) = 0.33 (0.11) (N = 21,127, k = 10), with values ranging from 0.20 (0.04) for education (N = 4,649, k = 10) to 0.55 
(0.30) for psychology (N = 461, k = 9). Multiple correlations and corresponding coefficients of determination for all eight 
of our exemplar program areas at the master’s level can be found in Table 17 (and for all master’s level program areas in 
the appendix). For programs at the doctoral level, the overall R (R 2 ) = 0.47 (0.22) (N = 4229, k = 10), with values ranging 
from 0.35 (0.12) for engineering ( N = 670, k = 6) to 0.52 (0.27) for mathematics and statistics (N = 122, k = 5). Multiple 
correlations and corresponding coefficients of determination for all seven of our exemplar program areas at the doctoral 
level can be found in Table 21 (and for all doctoral level program areas in the appendix). 

Note how these values for both master’s and doctoral programs greatly exceed the correlation between taking aspirin 
and reducing the risk of heart attack (0.02); between the effect of divorce and child well-being (0.09); and between the 
amount of brain tissue destruction and learning impairment in monkeys and humans (0.17). Many of these validity coef¬ 
ficients either exceed or are almost as large as the correlation between U.S. adult weight and height (0.44; Meyer et al., 
2001). Even multiple correlations unadjusted for range variation almost always exceed 0.20, and usually 0.30, especially 
for the prediction of GGPA in doctoral programs (see Tables 17 and 21). 

Admissions committees frequently use the GRE and UGPA in conjunction with each other in a compensatory manner 
(Powers, 2004; Walpole et al., 2002). A question that may arise is whether the GRE can add to prediction of graduate 
school success when admissions committees already possess applicants’ UGPAs. At both the master’s and doctoral degree 
levels, all GRE sections, together and individually, provide incremental validity above and beyond UGPA overall and 
across the reported program areas. This incremental prediction by the GRE occurs whether or not one adjusts for range 
variation. Although the increment is usually greater after adjusting for range variation, all of the values in Tables 17 and 
21 (adjusted for range variation) and Tables 16 and 20 (not adjusted for range variation) for “A R 2 over UGPA” exceed 
zero. That observation includes the increments in validity that the GRE Analytical Writing section used alone provides. 
The smallest unadjusted or adjusted value of “AR 2 over UGPA” anywhere among exemplar program areas is 0.01, which 
is equal to an incremental multiple correlation (AR) of 0.10. As discussed above, a zero-order correlation of only half of 
that size (i.e., 0.05) has substantial predictive value. Yet, this “AR 2 over UGPA” of 0.01 reflects predictive validity over 
and. above UGPA, and most of the values for “AR 2 over UGPA” are several times the size of 0.01. If one uses the overall, 
adjusted, doctoral value of 0.12 for “AR 2 over UGPA” in Table 21 (approximately equal to a AR value of 0.35), then, 
based on the conservative assumptions that we have previously made for public universities, the annual gain to an SUS 
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doctoral graduate program (without consideration of costs) from using optimally weighted sections of the GRE together 
(i.e.. Verbal Reasoning + Quantitative Reasoning + Analytical Writing) would be $103,054 over and above any financial 
gain from already using UGPA in predicting GGPA . 8 This incremental coefficient of determination of 0.12 is adjusted to 
account for multivariate range variation; if instead the unadjusted value of 0.15 (i.e., a AR value of approximately 0.39) 
was used (see Table 20), then the annual gain to an SUS doctoral graduate program (without consideration of costs) would 
be $114,871 over and above any financial gain from already using UGPA in predicting GGPA. 

In addition, the average contribution of the GRE sections to the amount of variability that they explain when used 
in conjunction with UGPA is substantial. For predicting GGPA in doctoral programs, GRE sections usually account for 
the majority of the variance that they and UGPA collectively explain. These findings hold whether or not one adjusts for 
the impact of range variation. In Tables 16 and 17 (master’s level programs) and Tables 20 and 21 (doctoral programs), 
%C X j +ugpa ( a percentage based on a usefulness weight from general dominance analysis; Budescu, 1993) represents the 
percentage of the R 2 that is uniquely attributable to a GRE section or UGPA, on average, when it might be considered in 
admissions decisions along with other predictors. On average signifies the fact that usefulness weights reflect R 2 from all 
possible regression submodels in addition to a regression model containing all variables. In Table 17 (usefulness weight 
percentages adjusted for range variation) and Table 16 (usefulness weight percentages not adjusted for range variation), 
%C X j +U GPA fi> r UGPA in predicting GGPA for master’s level programs is 56% (53% unadjusted) overall, with a minimum 
value of 32% (29% unadjusted) and a maximum value of 69% (67% unadjusted). On average, across master’s level pro¬ 
gram areas, GRE sections uniquely account for more than 40% of the variability in GGPA that GRE and UGPA collectively 
explain. At worst, GRE sections uniquely account, on average, for about one third of the variance in GGPA for a particular 
program area. At best, they account, on average, for about two thirds of it. The validity findings for the GRE sections in 
predicting GGPA are even stronger for doctoral level programs. In Table 21 (usefulness weight percentages adjusted for 
range variation) and Table 20 (usefulness weight percentages unadjusted for range variation), %C x j +UGPA for UGPA in 
predicting GGPA for doctoral level programs is 42% (40% unadjusted) overall, with a minimum value of 24% (23% unad¬ 
justed) and a maximum value of 67% (69% unadjusted). On average, across doctoral level program areas, they uniquely 
account for more than half of the variability in GGPA that GRE and UGPA collectively explain. At worst, GRE sections 
uniquely account on average for about one third for a particular program area for doctoral level programs and, at best, they 
uniquely account on average for about three quarters. Sometimes, the newer GRE Analytical Writing section is, among 
the GRE sections, the one to which the largest percentage of variance explained is attributable. 


GRE Quartile Comparisons 


An observer might question the statistical metrics and their adjustments that we have reported thus far. Underlying these 
metrics and adjustments are sensible, yet abstract, arguments (e.g., that adjustments for range variation are necessary 
so that a validity coefficient represents the population about which we want to draw a conclusion), statistical assump¬ 
tions that might not hold true (e.g., homoscedasticity), and mathematical operations that—while sound—might seem 
intricate and opaque to stakeholders such as admissions committees in nonmathematical disciplines (e.g., solving for 
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when adjusting for multivariate range variation). Consequently, we adopt the straightforward 
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methodology of Bridgeman et al. (2009) to report differences in specific graduate grade thresholds (i.e., the probability 
of achieving a graduate school grade of C+ or lower and the probability of achieving a GGPA of at least 3.8) for the top 
versus bottom GRE score quartiles for each GRE section. As mentioned, we use the previously discussed validity metrics 
and their adjustments because they account for the entire distribution of scores, can represent validity for the applicant 
pool, allow for multivariate comparisons for a single dataset (e.g., GRE sections vs. UGPA), and permit comparisons to 
similar validity metrics (usually zero-order correlations) that have been used in many past validity studies. 

Achieving a graduate school grade of C+ or lower can result in at least an academic warning from a graduate school 
or department. In Table 18 (master’s level programs) and Table 22 (doctoral level programs), all but two sets of low/high 
values for this criterion across GRE sections are uniformly greater than 1, indicating that enrollees who received scores 
in the lowest GRE quartile were more likely than enrollees who received scores in the highest GRE quartile to achieve 
a grade of C+ or lower. Note that these trends generally hold for the newer GRE Analytical Writing section as well. For 
master’s students overall, 26% of those who scored in the bottom quartile for GRE Analytical Writing achieved a graduate 
school grade of C+ or lower. Only 16% of those who scored in the top quartile for GRE Analytical Writing achieved this 
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same result. Therefore, 1.65 times as many students in the bottom quartile versus the top quartile (26%/16%) obtained a 
graduate school grade of C+ or lower. Findings for the probability of attaining a GGPA of at least 3.8 also show validity 
for the GRE sections, including the Analytical Writing section. Even with grade inflation, achieving a 3.8 or higher GGPA 
(at least close to straight As) might represent a noteworthy accomplishment. As shown for all GRE sections in Table 18 
(master’s level programs) and Table 22 (doctoral level programs), more students achieved at least a 3.8 GGPA who had 
scored in the top GRE quartile than in the bottom quartile (i.e., all high/low values exceed 1). For instance, doctoral 
students in biological and biomedical sciences who scored in the top quartile on the GRE Quantitative Reasoning section 
were 1.39 times more likely than those who scored in the bottom quartile to attain a 3.8 or greater GGPA (55% of top 
quartile/39% of the bottom quartile; see Table 22). One exception to general findings of low/high values larger than 1 is 
across GRE sections for doctoral programs in English language and literature/letters, because none of the 34 enrollees 
who scored in the lowest quartile on a GRE section received a graduate school grade of C+ or lower. The other exception 
is for the GRE Analytical Writing section for doctoral programs in mathematics and statistics. 


Conclusion 

Although the research establishing and reaffirming the generalizable validity of the GRE is compelling (Burton & Wang, 
2005; Kuncel, Hezlett, et al., 2001; Kuncel et al., 2010; Powers, 2004), our additional investigation builds upon it in impor¬ 
tant ways. We question the use of Cohen’s (1988) framework to determine whether a validity coefficient signifies the 
usefulness of an assessment. A strict set of rules for determining the importance of a zero-order correlation is convenient, 
especially if the correlation lacks context. Nevertheless, context does matter. Since validity information drives behaviors 
with major consequences, the fact that only a subset of stakeholders might have the expertise necessary to place a validity 
coefficient into context does not alone justify strict adherence to a set of perfunctory rules. Many biological and psycho¬ 
logical relationships (e.g., aspirin consumption and reduction in the risk of heart attack, destruction of brain tissue and 
learning impairment) that one might assume to be large by Cohen’s (1988) standards are, in fact, small to moderate in size. 
However, we believe that they are, nevertheless, meaningful and important in the sense that they should promote behav¬ 
iors that add value (e.g., encourage the use of aspirin to reduce the risk of heart attack, persuade graduate programs to 
require and use the GRE for admissions) and discourage behaviors that cause harm (e.g., dissuade parents from allowing 
their children to participate in activities that substantially risk brain injury, deter graduate programs from discontinuing 
use of the GRE for admissions). 

Zero-order correlations and alternative metrics to zero-order correlations demonstrate that the GRE sections, includ¬ 
ing the Analytical Writing section introduced in October 2002, provide value alone and in comparison to UGPA, the 
predictor with which GRE sections are often used by admissions committees in a compensatory manner. 9 Whether one 
looks at the multiple correlation for regression models of GRE sections (R), the incremental coefficient of determina¬ 
tion (AR 2 ) for GRE sections over and above UGPA, usefulness weight percentages (%C X) ), or GRE quartile comparisons 
(high/low or low/high), one observes abundant empirical evidence that the GRE sections provide value in decision mak¬ 
ing if the accurate prediction of GGPA is one’s objective. GGPA remains the most common measure of graduate student 
performance. 

In addition, we did not adjust any of the validity metrics that we reported (i.e., zero-order correlations, multiple corre¬ 
lations, incremental coefficients of multiple determination, usefulness weights, or quartile comparisons) for measurement 
error in the criterion. With an outcome measure of academic achievement more reliable than GGPA, we would have seen 
higher values for validity. However, those other measures were unavailable in our dataset, and, as already noted, GGPA 
remains the most commonly used outcome measure of graduate student success. Furthermore, and as has been the case 
for most predictive validity studies, we did not account for self-selection effects. In other words, the validity metrics that 
we provide are attenuated by the range restriction that results from people deciding not to even apply to a program area 
because their scores are too low. With the Law School Admission Test (LSAT) used as a comparison measure, research 
has demonstrated in the selection context that range restriction increases markedly when people know their scores in 
advance, as opposed to being ignorant about them (Kuncel & Klieger, 2007). Unlike the GRE test, admissions variables 
such as UGPA, personal statements, and letters of recommendation suffer from inflationary and leniency effects (V. John¬ 
son, 2003; Murphy et al., 2009; Vannelli et al., 2007). Consequently, we believe that range restriction due to self-selection 
effects would be larger for the GRE than for other admissions variables. 10 


36 


GRE Board Research Report No. 14-03 and ETS Research Report Series No. RR-14-26. © 2014 Educational Testing Service 


D. M. Klieger et al. 


New Perspectives for Predicting Graduate School Grades 


There are some limitations to our arguments and study. The seriousness of the consequences involved in other types 
of relationships that do not have large effect sizes by Cohen’s (1988) standards (i.e., heart attack, brain damage) is readily 
observable, precisely measurable, and practically indisputable. The seriousness of the consequences for failing to use the 
GRE is more opaque. Some graduate programs might be willing to confidentially provide remediation for those students 
who lack the ability to perform well, even though that remediation has costs. Alternatively, some graduate programs might 
surreptitiously adjust their outcome criterion standards and graduate into society students who lack KSAOs needed for 
success. Although these effects might go unseen and thus unmeasured, they remain real and become someone’s cost or 
problem. Furthermore, our utility analyses (based on Hunter & Hunter, 1984, and Taylor & Russell, 1939) rest on assump¬ 
tions that may be oversimplified or incorrect. In real-world contexts, graduate school (and workplace) performance is 
multidimensional and fluctuates over time, so a single cutoff score for criterion performance assumed by Taylor and Rus¬ 
sell (1939) is arguably inauthentic. In financial analyses of utility, the dollar value for a unit of improvement in criterion 
performance (c^,) usually is ambiguous. 

In addition, our statistical procedures for estimating validity (e.g., adjustment for multivariate range variation) depend 
on assumptions that might not hold true (e.g., that the relationship between the dependent and independent variables is 
linear, and the associated residuals are homoscedastic). We did not test linearity or homoscedasticity assumptions. Among 
other considerations, there is a very high risk of several false positives (250 tests * Type I error rate of 0.05 = 12.5). More¬ 
over, to account for the impact of range variation, we made univariate adjustments that do not require any assumptions 
about the relationships between independent variables, and they substantiate the predictive value of the GRE examina¬ 
tion. Furthermore, the quartile comparisons approach neither made nor required either a linearity or homoscedasticity 
assumption at all, and, as one saw, the results of the quartile comparisons approach evidenced GRE predictive validity. 
Additionally, our utility analyses showed substantial predictive value for the GRE test based on validity metrics that were 
not at all adjusted for the impact of range restriction. 

Although our alternatives to zero-order effect sizes do evidence predictive validity of the GRE, it is not mathematically 
necessary that they do so in other contexts (for other university systems, for different program areas, etc.). We cannot gen¬ 
eralize validity beyond the context for which we have data. Notwithstanding these concerns, we believe that our approaches 
were reasonable and would replicate in other contexts. Nevertheless, we firmly believe that additional investigation is nec¬ 
essary. Future validity studies should employ as many validity metrics as resources permit — such as multiple correlation 
for regression models (R), the incremental coefficient of determination (A R 2 ), usefulness weight percentages (%C x j), and 
quartile comparisons (high/low or low/high) — to ascertain predictive validity for specific contexts, as well as for validity 
generalization purposes. Given the compensatory nature of predictors in many situations, they should account for range 
variation in a multivariate way. Zero-order, incremental, and other validity information should be translated into—and 
then should be communicated in—language that decision makers who are less familiar with psychometrics and statistics 
can more readily understand. These translations could include how use of the GRE test (or any other assessment) affects 
the probability of admitted students’ success in graduate school, how much better admitted students will perform in grad¬ 
uate school if selected with the assessment (rather than without it), and the net financial value that use of the assessment 
provides to stakeholders. In the United States, one of the fundamental legal precepts regarding the worth of a piece of 
evidence is that “a brick is not a wall” (Advisory Comm.’s Notes to Federal Rule of Evidence § 401, quoting McCormick, 
Strong, & Broun, 1992, § 152, p. 317). We adopt the same perspective here. That is, we maintain that it is unreasonable to 
expect zero-order validity coefficients (especially based on an inflexible set of interpretative guidelines) to fully settle the 
question of whether an assessment demonstrates sufficient predictive value. Each additional analysis and argument adds 
another brick to the wall and can, thereby, increase our confidence about what a zero-order correlation tells us and does 
not tell us about the value of an assessment. 


Notes 

1 Includes tuition and fees, books and educational supplies, living expenses, and other education-related costs. 

2 The inclusion in this study of data for the new GRE Analytical Writing section is especially notable due to the lack of existing 
predictive validity evidence for this GRE section. The data analyzed in other studies (Burton & Wang, 2005; Kuncel, Hezlett, 

et al., 2001; Kuncel et al., 2010; Powers, 2004) were from GRE examinations taken prior to the introduction of the new Analytical 
Writing section in October, 2002. 
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3 As we discuss elsewhere, although GGPA is a highly imperfect assessment of student success, (a) it is likely to continue to be the 
most widely used measure of graduate student performance for the foreseeable future, and (b) we could not analyze other 
criterion outcomes (e.g., time to graduate degree completion, graduate school faculty ratings of graduate student performance, 
etc.) due to lack of clear information. We strongly encourage researchers and other stakeholders interested in GRE predictive 
validity to examine these alternatives to GGPA (as did Kuncel, Hezlett, et al., 2001; Kuncel et ah, 2010; Burton & Wang, 2005) 
because (a) they might be more reliable measures of academic achievement, and (b) they might measure aspects of graduate 
student success that GGPA does not. 

4 We did not employ hierarchical linear modeling (HLM) per se. (Correlation is a special case of regression, and, as we would have 
done with HLM, we identified separate subgroups whose lines of best fit might differ from those of other subgroups.) As we 
mentioned elsewhere, we selected the subgroups for which we separately report validity metrics (i.e., degree levels and program 
areas) based on a combination of what we thought would make theoretical sense as well as a desire to maintain reasonable sample 
sizes. Additional subgroup analyses were limited by considerations of sample size, as well as a lack of information in the SUS data 
that would allow us to create those subgroups. Furthermore, our methods avoid at least some of the problems associated with the 
use of regression weights (see our discussion in the section titled “Relative Importance/Usefulness and AR 2 for Enrolled Students 
and the Applicant Pool”). 

5 We did not test linearity or homoscedasticity assumptions. Please refer to our discussion of limitations in the “Conclusions” 
section. 

6 Similar formulations appear in Brogden (1949) and Cronbach and Gleser (1965). 

7 Had we reason to consider statistical shrinkage of multiple correlations in this study (and the impact on incremental coefficients 
of determination and usefulness weights), the effect of that shrinkage on values that we calculated would be minimal. This is 
partly due to the fact that our sample sizes (Ns) for exemplar groups defined by two-digit CIP codes within each degree level are 
at least 122. Shrinkage results from a kind of sampling error for the value of a regression-based multiple correlation, R, but its 
impact on the value of R is systematic in direction. As its name suggests, the effect of shrinkage is always to reduce the value of R. 
When a regression equation is reused on the population or another sample, it is not optimal, as it was for the sample on which it 
was based; hence, the multiple correlation for the population or the new sample is smaller than it was for the original sample. The 
size of the original multiple correlation (R), the size of the original sample (N), and the number of predictors (p) determine the 
amount of shrinkage (Einhorn & Hogarth, 1975). Across all multiple correlations for our exemplar program areas (whether or 
not adjusted for range variation and whether directly reported or used to calculate incremental coefficients of determination or 
usefulness weights), the worst-case shrinkage scenario in terms of relative (i.e., percentage) decrease in R would occur for 
doctoral programs in English language and literature/letters, where N = 137 and p = 3 (see Table 20). The amount of shrinkage in 
this case would reduce the multiple correlation from 0.37 (unadjusted for range variation) to 0.35 (based on formulae from 
Browne, 1975, and Cattin, 1980, that estimate shrunken R when the regression equation is reused on a new sample). 
Mathematically, the shrinkage effect from reuse of the regression equation on the population would be the same or smaller (see 
formula in Olkin & Pratt, 1958). 

8 We used AR as the value for rxy in the utility formula from Hunter and Hunter (1984). All other values remain the same. 

9 We were not surprised that the new GRE Analytical Writing section also demonstrated substantial predictive value. Based on 
interviews with subject matter experts and a literature review, Enright and Gitomer (1989) identified Explanation (which 
includes reasoning and the development of logical arguments) and Communication (which includes writing) as core 
competencies for success in graduate school. Furthermore, the “taxonomy of higher order performance components” that 
Campbell, McCloy, Oppler, and Sager (1993) developed based on research for the U.S. military included a written and oral 
communication task factor. Kuncel, Campbell, Hezlett, and Ones (2001), Reeve and Hakel (2001), and Oswald, Schmitt, Kim, 
Ramsay, & Gillespie (2004) have adapted the Campbell et al. (1993) taxonomy for use in higher education. 

10 Burton and Wang (2005) is one study that attempted to account for self-selection effects to at least some extent by adjusting for 
restriction in range using GRE test-taking populations as references. GRE examinees might not apply to a graduate or 
professional program for reasons unrelated to their GRE scores (e.g., cost, distance from family, research interests). Therefore, 
although it can be a difficult undertaking, we recommend that, for purposes of range variation adjustments that account for 
self-selection effects, researchers first try to carefully identify examinees who did not apply to a graduate program in question for 
reasons related to their GRE scores, rather than primarily due to other considerations. Otherwise, procedures that account for the 
effects of range variation might nontrivially overadjust or underadjust predictive validity metrics. 
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Appendix 

Univariate and Multivariate Predictive Validity Analyses of GRE-GGPA Relationships for All 

Program Areas 

Table Al Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for All 
Program Areas: Master’s Seekers 


General program area (with its 
2-digit CIP code) 

Overall 

Agriculture 
& related 
sciences (01) 

Natural 

resources 

& conservation 
(03) 

Architecture 
& related 
services (04) 

Area, ethnic, 
cultural, 

& gender 
studies (05) 

Communication, 

journalism, 

related 

programs (09) 

k (Total # of universities 

contributing data) 

10 

2 

7 

6 

5 

8 

N (Total # of students contributing data) 

21,127 

194 

165 

569 

144 

716 

Attenuation of p XY due 

Cronbach s alpha 

0.72 

0.73 

0.51 

0.77 

0.64 

0.70 

to unreliability of 
criterion (cumulative 
graduate GPA) 

Reduction in 
magnitude of p XY 

-15% 

-14% 

-28% 

-13% 

-20% 

-16% 

GRE Verbal Reasoning 

Mean 

1.12 

1.16 

1.15 

1.13 

1.24 

1.19 


^obs 

0.15 

0.19 

0.25 

0.24 

0.21 

0.17 


Grv 

0.16 

0.22 

0.28 

0.27 

0.25 

0.18 


r 

' mrv 

0.16 

0.27 

0.33 

0.24 

0.25 

0.17 


r mrv+UGPA 

0.17 

0.25 

0.35 

0.26 

0.25 

0.19 

GRE Quantitative 

Mean 

1.14 

1.29 

1.10 

1.15 

1.15 

1.22 

Reasoning 

^obs 

0.13 

0.14 

0.25 

0.19 

0.24 

0.23 


Grv 

0.15 

0.19 

0.26 

0.21 

0.28 

0.28 


r 

' mrv 

0.14 

0.22 

0.33 

0.20 

0.27 

0.24 


r mrv+UGPA 

0.15 

0.23 

0.34 

0.20 

0.26 

0.24 

GRE Analytical Writing 

Mean Urr 

1.05 

1.13 

1.16 

1.08 

1.11 

1.08 


^obs 

0.16 

0.26 

0.25 

0.18 

0.25 

0.20 


Grv 

0.17 

0.29 

0.30 

0.19 

0.27 

0.21 


r 

' mrv 

0.19 

0.32 

0.28 

0.18 

0.29 

0.21 


r mrv+UGPA 

0.19 

0.31 

0.27 

0.19 

0.28 

0.21 


Note. CIP = classification of instructional programs; GPA. 


Table A2 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for All 
Program Areas: Masters Seekers 


Engineering 

technologies Foreign Family & English 

& engineering languages, consumer language 

General program area Education Engineering related literature, & sciences/human & literature/ 

(with its 2-digit CIP code) (13) (14) fields (15) linguistics (16) sciences (19) letters (23) 


k (Total # of universities contributing data) 

10 

7 

2 

6 

2 

9 

N (Total # of students contributing data) 

4,649 

1,481 

141 

238 

87 

552 

Attenuation of p XY due to 

Cronbach s alpha 

0.70 

0.76 

0.80 

0.67 

0.66 

0.69 

unreliability of criterion 

Reduction in 

-17% 

-13% 

-11% 

-18% 

-19% 

-17% 

(cumulative graduate GPA) 
GRE Verbal 

magnitude of p XY 
Mean Urr 

1.08 

1.26 

1.32 

1.22 

1.17 

1.16 

Reasoning 

*”obs 

0.12 

0.10 

0.14 

0.19 

0.08 

0.26 


Grv 

0.13 

0.12 

0.17 

0.22 

0.09 

0.29 


r 

' mrv 

0.13 

0.12 

0.15 

0.19 

0.04 

0.30 


r mrv+UGPA 

0.15 

0.14 

0.14 

0.18 

0.07 

0.32 

GRE Quantitative 

Mean Uj^ 

1.10 

1.06 

1.25 

1.11 

1.45 

1.07 

Reasoning 

*"obs 

0.09 

0.13 

0.33 

0.16 

-0.04 

0.17 


Grv 

0.10 

0.13 

0.39 

0.17 

-0.08 

0.18 


r 

' mrv 

0.10 

0.14 

0.44 

0.20 

-0.10 

0.15 


r mrv+UGPA 

0.12 

0.14 

0.43 

0.19 

-0.06 

0.16 

GRE Analytical 

Mean Urr 

1.04 

1.04 

0.92 

1.16 

1.00 

1.06 

Writing 

^obs 

0.14 

0.14 

0.20 

0.19 

0.08 

0.28 

Grv 

0.15 

0.15 

0.19 

0.21 

0.09 

0.29 


r 

' mrv 

0.15 

0.15 

0.22 

0.16 

0.08 

0.33 


r mrv+UGPA 

0.16 

0.16 

0.21 

0.13 

0.05 

0.33 


Note. CIP = classification of instructional programs. 
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Table A3 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for All 
Program Areas: Masters Seekers 


General program area 
(with its 2-digit CIP code) 

Liberal arts 
& sciences, 
general studies 
& humanities 
(24) 

Library 

science 

(25) 

Biological 
& biomedical 
sciences 
(26) 

Mathematics 
& statistics 
(27) 

Multi/ 

interdisciplinary 

studies 

(30) 

Parks, 
recreation, 
leisure & 
fitness 
studies (31) 

k (Total # of universities contributing data) 

6 

2 

9 

8 

4 

6 

N (Total # of students contributing data) 

118 

867 

445 

230 

75 

375 

Attenuation of p XY due to 

Cronbachs alpha 

0.70 

0.73 

0.64 

0.71 

0.68 

0.70 

unreliability of criterion 

Reduction in 

-17% 

-15% 

-20% 

-16% 

-17% 

-16% 

(cumulative graduate 

magnitude of p XY 







GPA) 








GRE Verbal Reasoning 

Mean Ujm 

1.01 

1.08 

1.10 

1.14 

1.14 

1.13 


^"obs 

0.12 

0.06 

0.13 

0.06 

0.19 

0.07 


r 

' urv 

0.13 

0.07 

0.13 

0.06 

0.22 

0.07 


r 

' mrv 

0.12 

0.08 

0.18 

0.05 

0.10 

0.08 


r mrv+UGPA 

0.12 

0.09 

0.20 

0.12 

0.09 

0.11 

GRE Quantitative 

Mean U RR 

1.01 

1.06 

1.16 

1.38 

1.10 

1.29 

Reasoning 

^obs 

0.04 

0.05 

0.19 

0.16 

0.28 

0.10 


r 

urv 

0.04 

0.05 

0.22 

0.22 

0.28 

0.13 


r 

' mrv 

0.05 

0.05 

0.17 

0.25 

0.09 

0.12 


r mrv+UGPA 

0.05 

0.07 

0.15 

0.21 

0.10 

0.15 

GRE Analytical Writing 

Mean Uj^ 

1.02 

1.06 

1.06 

1.07 

1.00 

1.12 


^obs 

0.08 

0.17 

0.15 

0.11 

0.29 

0.14 


r 

' urv 

0.08 

0.18 

0.15 

0.12 

0.30 

0.15 


r 

' mrv 

0.16 

0.18 

0.19 

0.08 

0.07 

0.16 


r mrv+UGPA 

0.16 

0.19 

0.20 

0.17 

0.06 

0.16 


Note. CIP = classification of instructional programs. 


Table A4 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for All 
Program Areas: Masters Seekers 


General program area 
(with its 2-digit CIP code) 

Philosophy 
& religious 
studies (38) 

Physical 
sciences (40) 

Psychology 

(42) 

Security 
& protective 
services (43) 

Public 

administration 
& social 
service 

professions (44) 

k (Total # of universities contributing data) 

5 

7 

9 

7 

9 

N (Total # of students contributing data) 

81 

240 

461 

469 

1,713 

Attenuation of p XY due to 

Cronbach s alpha 

0.74 

0.67 

0.69 

0.65 

0.74 

unreliability of criterion 

Reduction in 

-14% 

-18% 

-17% 

-19% 

-14% 

(cumulative graduate GPA) 
GRE Verbal Reasoning 

magnitude of p XY 
Mean U RR 

1.23 

1.30 

1.24 

1.05 

1.11 


^obs 

0.04 

0.14 

0.07 

0.16 

0.16 


r 

' urv 

0.05 

0.16 

0.09 

0.17 

0.18 


r 

mrv 

-0.01 

0.04 

0.08 

0.18 

0.17 


r mrv+UGPA 

-0.02 

0.11 

0.09 

0.18 

0.19 

GRE Quantitative Reasoning 

Mean U RR 

1.02 

1.23 

1.24 

1.18 

1.11 


*"obs 

0.13 

0.05 

0.09 

0.04 

0.18 


Grv 

0.13 

0.04 

0.11 

0.04 

0.20 


r 

' mrv 

-0.04 

0.09 

-0.02 

0.07 

0.19 


r mrv+UGPA 

-0.07 

0.09 

-0.01 

0.07 

0.21 

GRE Analytical Writing 

Mean U RR 

1.10 

1.13 

1.18 

1.02 

1.03 


*"obs 

0.11 

0.23 

0.19 

0.21 

0.18 


r 

' urv 

0.10 

0.25 

0.22 

0.21 

0.18 


r 

mrv 

0.19 

0.24 

0.31 

0.26 

0.18 


r mrv+UGPA 

0.18 

0.26 

0.23 

0.27 

0.20 


Note. CIP = classification of instructional programs. 
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Table A5 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for All 
Program Areas: Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

Social 

sciences 

(45) 

Visual & 
performing 
arts (50) 

Health 

professions & 
clinical 
sciences (51) 

Business, 

management, 

marketing 

(52) 

History 

(54) 

k (Total # of universities contributing data) 

9 

6 

10 

9 

8 

N (Total # of students contributing data) 

1,133 

744 

3,772 

901 

289 

Attenuation of p XY due to 

Cronbach’s alpha 

0.74 

0.76 

0.76 

0.71 

0.75 

unreliability of criterion 

Reduction in 

-14% 

-13% 

-13% 

-16% 

-14% 

(cumulative graduate GPA) 
GRE Verbal Reasoning 

magnitude of p XY 
Mean U n 

1.13 

1.06 

1.09 

1.14 

1.16 


r obs 

0.21 

0.12 

0.18 

0.11 

0.14 


Grv 

0.24 

0.13 

0.19 

0.13 

0.15 


r 

0.27 

0.11 

0.20 

0.13 

0.14 


r mrv+UGPA 

0.28 

0.11 

0.22 

0.14 

0.16 

GRE Quantitative Reasoning 

Mean U RR 

1.16 

1.09 

1.18 

1.17 

1.12 


r obs 

0.15 

0.10 

0.15 

0.11 

0.18 


Grv 

0.18 

0.11 

0.17 

0.13 

0.20 


r 

0.19 

0.09 

0.18 

0.13 

0.15 


r mrv+UGPA 

0.21 

0.09 

0.21 

0.15 

0.17 

GRE Analytical Writing 

Mean U RR 

1.06 

1.05 

1.04 

1.05 

1.07 


r obs 

0.20 

0.11 

0.16 

0.11 

0.11 


Grv 

0.21 

0.12 

0.16 

0.12 

0.11 


r 

0.21 

0.12 

0.18 

0.13 

0.11 


r mrv+UGPA 

0.21 

0.12 

0.20 

0.15 

0.15 


Note. CIP = classification of instructional programs. 


Table A6 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Master’s 
Seekers 


Area, ethnic, Communication, 


General program area 
(with its 2-digit CIP code) 

Overall 

Agriculture 
& related 
sciences 
(01) 

Natural 
resources & 
conservation 
(03) 

Architecture & 
related 
services (04) 

cultural, & 
gender 
studies 
(05) 

journalism, 

related 

programs 

(09) 

k (Total # of universities contributing data) 

10 

2 

7 

6 

5 

8 

N (Total # of students contributing data) 

21,127 

194 

165 

569 

144 

716 

Model R (R 2 ) 

R (R 2 ) 

0.27 (0.07) 

0.35 (0.13) 

0.47 (0.23) 

0.30 (0.09) 

0.37 (0.14) 

0.37 (0.14) 


R+ugpa (R 2 +ugpa) 

0.36 (0.13) 

0.44 (0.19) 

0.54 (0.29) 

0.42 (0.17) 

0.44 (0.20) 

0.47 (0.22) 

AR 2 over UGPA 

GRE V + Q + AW 

0.07 

0.07 

0.19 

0.08 

0.13 

0.10 


GRE Verbal Reasoning 

0.03 

0.02 

0.08 

0.05 

0.02 

0.05 


GRE Quantitative 
Reasoning 

0.02 

0.02 

0.07 

0.04 

0.06 

0.04 


GRE Analytical 

Writing 

0.03 

0.05 

0.07 

0.03 

0.06 

0.03 

GRE Verbal 

P 

0.07 

0.19 

0.10 

0.14 

0.07 

-0.04 

Reasoning 

P+XSGVh 

0.07 

0.13 

0.12 

0.13 

0.04 

0.06 


%c xj 

30% 

25% 

22% 

41% 

27% 

25% 


%C X j +UGPA 

14% 

11% 

17% 

18% 

14% 

15% 

GRE Quantitative 

p 

0.08 

-0.01 

0.17 

0.11 

0.16 

0.29 

Reasoning 

P +XJGVA 

0.06 

0.02 

0.10 

0.10 

0.14 

0.14 


%C xj 

28% 

13% 

42% 

26% 

33% 

41% 


%C X j +UGPA 

14% 

8% 

25% 

15% 

24% 

19% 
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Table A6 Continued 


General program area 
(with its 2-digit CIP code) 

Overall 

Agriculture 
& related 
sciences 
(01) 

Natural 
resources 8t 
conservation 
(03) 

Architecture 8t 
related 
services (04) 

Area, ethnic, 
cultural, & 
gender 
studies 
(05) 

Communication, 

journalism, 

related 

programs 

(09) 

GRE Analytical /? 

0.12 

0.18 

0.19 

0.08 

0.18 

0.18 

Writing P+vgpa 

0.10 

0.14 

0.16 

0.06 

0.15 

0.11 

%c xj 

40% 

58% 

34% 

29% 

37% 

32% 

^Qtj+UGPA 

16% 

29% 

25% 

10% 

25% 

14% 

UGPA p 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

P+XJGVA 

0.23 

0.27 

0.27 

0.29 

0.24 

0.30 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

^Otj+UGPA 

53% 

47% 

30% 

53% 

33% 

50% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V =Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A7 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Master’s 
Seekers 


General program area 
(with its 2-digit CIP code) 

Computer 8t 
information 
sciences 
(11) 

Education 

(13) 

Engineering 

(14) 

Engineering 
technology 8t 
engineering 
related 
fields (15) 

Foreign 
languages, 
literature, 8t 
linguistics (16) 

Family & 
consumer 

sciences/ 
human 
sciences (19) 

k (Total # of universities contributing data) 

8 

10 

7 

2 

6 

2 

N (Total # of students 

; contributing data) 

278 

4,649 

1,481 

141 

238 

87 

Model R (R 2 ) 

R (R 2 ) 

0.37 (0.14) 

0.19 (0.04) 

0.21 (0.04) 

0.38 (0.14) 

0.32 (0.10) 

0.21 (0.04) 


R+ugpa (^ 2 +ugpa) 

0.50 (0.25) 

0.30 (0.09) 

0.34 (0.12) 

0.50 (0.25) 

0.38 (0.15) 

0.42 (0.18) 

AR 2 over UGPA 

GRE V + Q + AW 

0.17 

0.03 

0.03 

0.12 

0.10 

0.05 


GRE Verbal 

Reasoning 

0.08 

0.01 

0.01 

0.02 

0.03 

0.02 


GRE Quantitative 
Reasoning 

0.05 

0.01 

0.01 

0.10 

0.01 

0.02 


GRE Analytical 
Writing 

0.05 

0.02 

0.01 

0.03 

0.02 

0.01 

GRE Verbal 

P 

0.08 

0.06 

0.02 

0.02 

0.10 

0.09 

Reasoning 

P+VGPA 

0.08 

0.07 

0.03 

0.00 

0.09 

0.06 


%C X) 

33% 

28% 

20% 

5% 

32% 

19% 


%C X j +UGPA 

21% 

12% 

7% 

3% 

23% 

7% 

GRE Quantitative 

p 

0.16 

0.03 

0.10 

0.30 

0.15 

-0.10 

Reasoning 

P +UGPA 

0.13 

0.02 

0.05 

0.29 

0.13 

-0.19 


%C XJ 

40% 

21% 

36% 

71% 

17% 

62% 


%C X j +UGPA 

18% 

9% 

14% 

35% 

8% 

10% 

GRE Analytical 

p 

0.13 

0.11 

0.11 

0.12 

0.03 

0.08 

Writing 

P+ UGPA 

0.11 

0.08 

0.08 

0.08 

0.00 

0.02 


%C XJ 

25% 

47% 

43% 

22% 

48% 

21% 


%C X j+u G p A 

12% 

13% 

11% 

7% 

25% 

5% 

UGPA 

P 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


P+ UGPA 

0.27 

0.23 

0.27 

0.26 

0.18 

0.39 


%C XJ 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


%C X j+UGPA 

46% 

64% 

67% 

52% 

39% 

78% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 
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Table A8 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Master’s 
Seekers 


General program area 
(with its 2-digit CIP code) 

English 
language & 
literature/ 
letters (23) 

Liberal arts & 
sciences, 
general 
studies & 
humanities (24) 

Library 

science 

(25) 

Biological & 
biomedical 
sciences (26) 

Mathematics 
& statistics 
(27) 

Multi/ 

interdisciplinary 
studies (30) 

k (Total # of universities contributing data) 

9 

6 

2 

9 

8 

4 

N (Total # of students contributing data) 

552 

118 

867 

445 

230 

75 

Model R (R 2 ) 

R (R 2 ) 

0.37 (0.14) 

0.38 (0.14) 

0.18 (0.03) 

0.33 (0.11) 

0.35 (0.12) 

0.55 (0.30) 


R+ugpa (^ 2 +ugpa) 

0.43 (0.19) 

0.50 (0.25) 

0.23 (0.05) 

0.42 (0.18) 

0.48 (0.23) 

0.56 (0.31) 

AR 2 over UGPA 

GRE V + Q + AW 

0.13 

0.21 

0.03 

0.10 

0.15 

0.29 


GRE Verbal 

Reasoning 

0.07 

0.04 

0.00 

0.04 

0.05 

0.04 


GRE Quantitative 
Reasoning 

0.02 

0.05 

0.00 

0.04 

0.05 

0.14 


GRE Analytical 
Writing 

0.07 

0.07 

0.02 

0.03 

0.03 

0.07 

GRE Verbal 

P 

0.17 

0.34 

-0.01 

-0.01 

-0.03 

0.00 

Reasoning 

P+ UGPA 

0.17 

0.14 

-0.01 

-0.02 

0.04 

0.07 


%c xj 

37% 

53% 

7% 

31% 

27% 

17% 


%C x j+UGPA 

30% 

20% 

5% 

18% 

13% 

15% 

GRE Quantitative 

P 

0.02 

-0.37 

-0.01 

0.21 

0.30 

0.19 

Reasoning 

P+ UGPA 

0.01 

-0.25 

0.00 

0.18 

-0.02 

0.15 


%c xj 

14% 

12% 

4% 

41% 

40% 

43% 


%Cxj+UGPA 

9% 

10% 

3% 

18% 

15% 

41% 

GRE Analytical 

P 

0.22 

0.15 

0.18 

0.12 

-0.07 

0.12 

Writing 

P+ UGPA 

0.19 

0.28 

0.16 

0.09 

0.12 

0.04 


%c xj 

47% 

33% 

88% 

27% 

31% 

37% 


%C x j+ UG PA 

29% 

27% 

49% 

15% 

16% 

35% 

UGPA 

p 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


P+ UGPA 

0.18 

0.06 

0.14 

0.24 

0.34 

-0.03 


%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


%Cxj+UGPA 

29% 

43% 

43% 

46% 

56% 

5% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A9 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Master’s 
Seekers 


Parks, 


General program area 
(with its 2-digit CIP code) 

recreation, 
leisure, & 
fitness studies 
(31) 

Philosophy & 
religious 
studies (38) 

Physical 

sciences 

(40) 

Psychology 

(42) 

Security 8t 
protective 
services (43) 

Public 

administration 
& social service 
professions (44) 

k (Total # of universities contributing data) 

6 

5 

7 

9 

7 

9 

N (Total # of students contributing data) 

375 

81 

240 

461 

469 

1,713 

Model R (R 2 ) 

R (R 2 ) 

0.24 (0.06) 

0.47 (0.22) 

0.4 (0.16) 

0.35 (0.12) 

0.34 (0.12) 

0.24 (0.06) 


R+ugpa (^ 2 +ugpa) 

0.36 (0.13) 

0.52 (0.27) 

0.54 (0.29) 

0.43 (0.19) 

0.42 (0.17) 

0.34 (0.11) 

AR 2 over UGPA 

GRE V + Q + AW 

0.06 

0.20 

0.14 

0.14 

0.10 

0.05 


GRE Verbal 

Reasoning 

0.02 

0.04 

0.03 

0.02 

0.04 

0.02 


GRE Quantitative 
Reasoning 

0.02 

0.06 

0.05 

0.06 

0.02 

0.03 


GRE Analytical 
Writing 

0.02 

0.06 

0.04 

0.05 

0.05 

0.02 
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Table A9 Continued 


General program area 
(with its 2-digit CIP code) 

Parks, 
recreation, 
leisure, & 
fitness studies 
(31) 

Philosophy & 
religious 
studies (38) 

Physical 

sciences 

(40) 

Psychology 

(42) 

Security 8t 
protective 
services (43) 

Public 

administration 
8t social service 
professions (44) 

GRE Verbal 

P 

0.03 

-0.12 

0.03 

0.02 

0.06 

0.06 

Reasoning 

P+VGPA 

0.03 

-0.19 

-0.03 

0.05 

0.07 

0.04 


%c xj 

25% 

29% 

26% 

15% 

48% 

27% 


^Qq+UGPA 

9% 

22% 

11% 

11% 

27% 

13% 

GRE Quantitative 

p 

0.06 

0.03 

-0.01 

0.02 

0.00 

0.11 

Reasoning 

P+ UGPA 

0.06 

-0.01 

-0.01 

0.00 

-0.04 

0.11 


%c xj 

37% 

29% 

36% 

37% 

13% 

37% 


%Cxj+UGPA 

12% 

24% 

19% 

26% 

8% 

17% 

GRE Analytical 

P 

0.14 

0.20 

0.23 

0.20 

0.21 

0.11 

Writing 

P+ UGPA 

0.10 

0.31 

0.23 

0.20 

0.17 

0.09 


%c xj 

36% 

41% 

38% 

47% 

36% 

30% 


^Cxj+UGPA 

13% 

34% 

14% 

25% 

21% 

11% 

UGPA 

P 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


P+ UGPA 

0.26 

0.13 

0.34 

0.20 

0.23 

0.23 


%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


^Cxj+UGPA 

65% 

18% 

55% 

37% 

41% 

54% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A10 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Masters 
Seekers 


Health 


General program area 
(with its 2-digit CIP code) 

Social 

science 

(45) 

Visual & 
performing 
arts (50) 

professions 

8t clinical 
science (51) 

Business, 
management, 
marketing (52) 

History (54) 

k (Total # of universities 

contributing data) 

9 

6 

10 

9 

8 

N (Total # of students contributing data) 

1,133 

744 

3,772 

901 

289 

Model R (R 2 ) 

R (R 2 ) 

0.31 (0.10) 

0.22 (0.05) 

0.24 (0.06) 

0.23 (0.05) 

0.34 (0.12) 


R+UGPA (^ +ugpa) 

0.40 (0.16) 

0.28 (0.08) 

0.35 (0.12) 

0.32 (0.10) 

0.52 (0.27) 

AR 2 over UGPA 

GRE V + Q + AW 

0.08 

0.05 

0.06 

0.06 

0.15 


GRE Verbal Reasoning 

0.04 

0.02 

0.03 

0.02 

0.03 


GRE Quantitative Reasoning 

0.03 

0.02 

0.03 

0.02 

0.04 


GRE Analytical Writing 

0.02 

0.01 

0.02 

0.02 

0.05 

GRE Verbal Reasoning 

P 

0.13 

0.06 

0.11 

0.06 

0.06 


P+ UGPA 

0.11 

0.05 

0.10 

0.06 

-0.01 


%C XJ 

35% 

22% 

41% 

28% 

42% 


%C X j +UGPA 

17% 

14% 

18% 

18% 

19% 

GRE Quantitative 

p 

0.08 

0.05 

0.08 

0.07 

0.14 

Reasoning 

P+ UGPA 

0.08 

0.05 

0.09 

0.06 

0.15 


%c xj 

24% 

20% 

28% 

37% 

33% 


%C X j+ UG PA 

14% 

14% 

17% 

16% 

15% 

GRE Analytical Writing 

p 

0.12 

0.07 

0.10 

0.09 

0.07 


P+ UGPA 

0.08 

0.04 

0.09 

0.07 

0.07 


%C X) 

38% 

57% 

29% 

33% 

23% 


^Cxj+UGPA 

13% 

24% 

13% 

16% 

14% 

UGPA 

p 

N/A 

N/A 

N/A 

N/A 

N/A 


P+ UGPA 

0.25 

0.17 

0.24 

0.20 

0.36 


%C XJ 

N/A 

N/A 

N/A 

N/A 

N/A 


%C X j+UGPA 

52% 

47% 

50% 

48% 

49% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 
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Table All Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

Overall 

Agriculture 
& related 
sciences 
(01) 

Natural 
resources & 
conservation 
(03) 

Architecture 
& related 
services 
(04) 

Area, ethnic, 
cultural, & 
gender studies 
(05) 

Communication, 

journalism, 

related 

programs (09) 

k (Total # of universities contributing data) 

10 

2 

7 

6 

5 

8 

N (Total # of students contributing data) 

21,127 

194 

165 

569 

144 

716 

Model R (R 2 ) 

R (R 2 ) 

0.33 (0.11) 

0.39 (0.16) 

0.55 (0.3) 

0.31 (0.1) 

0.4(0.16) 

0.4 (0.16) 


R+ugpa (^~+ugpa) 

0.43 (0.19) 

0.46 (0.21) 

0.63 (0.39) 

0.44 (0.19) 

0.46 (0.21) 

0.48 (0.23) 

AR 2 over UGPA 

GRE V + Q + AW 

0.06 

0.08 

0.20 

0.07 

0.13 

0.10 


GRE Verbal 

Reasoning 

0.03 

0.04 

0.08 

0.05 

0.03 

0.05 


GRE Quantitative 
Reasoning 

0.03 

0.04 

0.09 

0.04 

0.08 

0.05 


GRE Analytical 
Writing 

0.03 

0.06 

0.10 

0.03 

0.07 

0.04 

GRE Verbal 

%c X) 

29% 

22% 

18% 

40% 

26% 

25% 

Reasoning 

^Cxj+UGPA 

14% 

9% 

15% 

17% 

16% 

15% 

GRE Quantitative 

%C X) 

31% 

14% 

42% 

27% 

35% 

48% 

Reasoning 

%C x j+UGPA 

14% 

11% 

22% 

14% 

23% 

19% 

GRE Analytical 

%c X) 

41% 

63% 

40% 

33% 

38% 

27% 

Writing 

%C X j +UGPA 

17% 

32% 

30% 

10% 

25% 

12% 

UGPA 

%C X) 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


%C x j +UGPA 

56% 

48% 

33% 

59% 

37% 

54% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A12 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

Computer & 
information 
sciences (11) 

Education 

(13) 

Engineering 

(14) 

Engineering 
technologies & 
engineering 
related fields 
(15) 

Foreign 
languages, 
literature, & 
linguistics 
(16) 

Family & 
consumer 

sciences/human 
sciences (19) 

k (Total # of universities contributing data) 

8 

10 

7 

2 

6 

2 

N (Total # of students contributing data) 

278 

4,649 

1,481 

141 

238 

87 

Model R (R 2 ) 

R (R 2 ) 

0.36 (0.13) 

0.20 (0.04) 

0.22 (0.05) 

0.47 (0.22) 

0.38 (0.15) 

0.22 (0.05) 


R+ugpa (^ 2 +ugpa) 

0.48 (0.23) 

0.32 (0.10) 

0.37 (0.14) 

0.54 (0.29) 

0.46 (0.21) 

0.5 (0.25) 

AR 2 over UGPA 

GRE V + Q + AW 

0.11 

0.03 

0.03 

0.14 

0.09 

0.07 


GRE Verbal Reasoning 

0.06 

0.02 

0.01 

0.01 

0.03 

0.02 


GRE Quantitative 
Reasoning 

0.05 

0.01 

0.02 

0.13 

0.02 

0.05 


GRE Analytical 

Writing 

0.04 

0.02 

0.01 

0.03 

0.03 

0.01 

GRE Verbal 

%c xj 

31% 

23% 

21% 

2% 

30% 

19% 

Reasoning 

^Otj+UGPA 

20% 

11% 

8% 

2% 

26% 

6% 

GRE Quantitative 

%c xj 

52% 

24% 

39% 

82% 

18% 

66% 

Reasoning 

%C x j+UGPA 

21% 

8% 

13% 

42% 

8% 

12% 

GRE Analytical 

%c xj 

17% 

54% 

40% 

16% 

52% 

15% 

Writing 

%C x j +UGPA 

10% 

13% 

10% 

6% 

26% 

3% 

UGPA 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


%C X j+ UG PA 

49% 

67% 

69% 

50% 

40% 

80% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


GRE Board Research Report No. 14-03 and ETS Research Report Series No. RR-14-26. © 2014 Educational Testing Service 


47 








D. M. Klieger etal. 


New Perspectives for Predicting Graduate School Grades 


Table A13 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

English 
language & 
literature/ 
letters 
(23) 

Liberal arts & 
sciences, 
general 
studies & 
humanities 
(24) 

Library 

science 

(25) 

Biological & 
biomedical 
sciences 
(26) 

Mathematics 8t 
statistics 
(27) 

Multi/ 

interdisciplinary 
studies (30) 

k (Total # of universities contributing data) 

9 

6 

2 

9 

8 

4 

N (Total # of students contributing data) 

552 

118 

867 

445 

230 

75 

Model R (R 2 ) 

R(R 2 ) 

0.41 (0.17) 

0.23 (0.05) 

0.19 (0.04) 

0.36 (0.13) 

0.42 (0.18) 

0.64 (0.41) 


R+UGPA +ugpa) 

0.47 (0.22) 

0.26 (0.07) 

0.25 (0.06) 

0.45 (0.21) 

0.55 (0.3) 

0.65 (0.42) 

A R 2 over UGPA 

GRE V + Q + AW 

0.12 

0.11 

0.03 

0.09 

0.13 

0.23 


GRE Verbal 

Reasoning 

0.07 

0.01 

0.01 

0.03 

0.03 

0.01 


GRE Quantitative 
Reasoning 

0.02 

0.02 

0.00 

0.04 

0.06 

0.16 


GRE Analytical 
Writing 

0.07 

0.03 

0.03 

0.04 

0.03 

0.06 

GRE Verbal 

%c xj 

40% 

58% 

10% 

32% 

22% 

14% 

Reasoning 

%Cxj+UGPA 

31% 

18% 

7% 

18% 

12% 

14% 

GRE Quantitative 

%c xj 

10% 

8% 

3% 

42% 

47% 

51% 

Reasoning 

^Cxj+UGPA 

7% 

7% 

2% 

18% 

21% 

46% 

GRE Analytical 

°/°C xj 

50% 

34% 

87% 

26% 

31% 

36% 

Writing 

^Cxj+UGPA 

30% 

27% 

48% 

16% 

17% 

37% 

UGPA 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


^Cxj+UGPA 

32% 

48% 

44% 

48% 

51% 

3% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A14 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

Parks, 
recreation, 
leisure, 8t 
fitness studies 
(31) 

Philosophy 8t 
religious 
studies 
(38) 

Physical 

sciences 

(40) 

Psychology 

(42) 

Security 8t 
protective 
services 
(43) 

Public 

administration & 
social service 
professions (44) 

k (Total # of universities 

contributing data) 

6 

5 

7 

9 

7 

9 

N (Total # of students contributing data) 

375 

81 

240 

461 

469 

1,713 

Model R (R 2 ) 

R (R 2 ) 

0.25 (0.06) 

0.58 (0.33) 

0.5 (0.25) 

0.55 (0.30) 

0.38 (0.14) 

0.26 (0.07) 


R+ugpa (^ 2 +ugpa) 

0.4 (0.16) 

0.63 (0.4) 

0.6 (0.36) 

0.65 (0.43) 

0.45 (0.2) 

0.36 (0.13) 

AR 2 over UGPA 

GRE V + Q + AW 

0.06 

0.24 

0.12 

0.13 

0.09 

0.05 


GRE Verbal Reasoning 

0.02 

0.02 

0.12 

0.03 

0.04 

0.03 


GRE Quantitative 
Reasoning 

0.03 

0.07 

0.06 

0.07 

0.01 

0.03 


GRE Analytical 

Writing 

0.03 

0.11 

0.03 

0.07 

0.05 

0.02 

GRE Verbal Reasoning 

%c xj 

20% 

30% 

30% 

16% 

56% 

23% 


^Otj+UGPA 

7% 

26% 

11% 

13% 

21% 

12% 

GRE Quantitative 

%c xj 

43% 

25% 

37% 

35% 

7% 

43% 

Reasoning 

^Cxj+UGPA 

14% 

23% 

21% 

22% 

4% 

17% 

GRE Analytical 

%c xj 

37% 

45% 

33% 

48% 

37% 

35% 

Writing 

^Cxj+UGPA 

14% 

44% 

14% 

23% 

20% 

15% 

UGPA 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


^Otj+UGPA 

65% 

7% 

53% 

41% 

54% 

56% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 
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Table A15 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

Social 

sciences 

(45) 

Visual 8c 
performing 
arts (50) 

Health 

professions 8c 
clinical 
sciences (51) 

Business, 

management, 

marketing 

(52) 

History 

(54) 

k (Total # of universities contributing data) 

9 

6 

10 

9 

8 

N (Total # of students contributing data) 

1,133 

744 

3,772 

901 

289 

Model R (R 2 ) 

R (R 2 ) 

0.38 (0.15) 

0.22 (0.05) 

0.28 (0.08) 

0.25 (0.06) 

0.37 (0.14) 


R+UGPA (^ +ugpa) 

0.46 (0.21) 

0.28 (0.08) 

0.40 (0.16) 

0.37 (0.13) 

0.54 (0.29) 

AR 2 over UGPA 

GRE V + Q + AW 

0.08 

0.05 

0.06 

0.05 

0.13 


GRE Verbal Reasoning 

0.05 

0.03 

0.04 

0.02 

0.04 


GRE Quantitative Reasoning 

0.04 

0.03 

0.04 

0.02 

0.04 


GRE Analytical Writing 

0.03 

0.01 

0.03 

0.02 

0.05 

GRE Verbal Reasoning 

%c xj 

35% 

23% 

40% 

32% 

42% 


%C X j +UGPA 

19% 

14% 

16% 

21% 

19% 

GRE Quantitative Reasoning 

%c xj 

24% 

22% 

34% 

37% 

36% 


%C x j +UGPA 

15% 

14% 

17% 

15% 

16% 

GRE Analytical Writing 

%c xj 

41% 

55% 

26% 

31% 

22% 


%C x j +UGPA 

16% 

22% 

12% 

17% 

14% 

UGPA 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 


%C X j +UGPA 

50% 

50% 

55% 

47% 

51% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A16 GRE Quartile Comparisons: Master’s Seekers 


Area, ethnic, Communication, 


General program area 
(with its 2-digit CIP code) 

Overall 

Agriculture 8c 
related 
sciences 
(01) 

Natural 
resources 8c 
conservation 
(03) 

Architecture 8c 
related 
services 
(04) 

cultural, 8c 
gender 
studies 
(05) 

journalism, 

related 

programs 

(09) 

k (Total # of universities contributing data) 

10 

2 

7 

6 

5 

8 

N (Total # of students contributing data) 

21,127 

194 

165 

569 

144 

716 


Probability of grade of C+ or lower 




GRE Verbal Reasoning Low quartile 

25% 

36% 

19% 

37% 

20% 

21% 

High quartile 

16% 

15% 

12% 

13% 

8% 

15% 

Low/high 

1.57 

2.37 

1.57 

2.83 

2.59 

1.41 

GRE Quantitative Reasoning Low quartile 

25% 

18% 

22% 

37% 

20% 

30% 

High quartile 

17% 

18% 

10% 

21% 

5% 

15% 

Low/high 

1.50 

0.97 

2.34 

1.75 

3.59 

2.03 

GRE Analytical Writing Low quartile 

26% 

26% 

11% 

42% 

20% 

22% 

High quartile 

16% 

12% 

2% 

22% 

5% 

16% 

Low/high 

1.65 

2.11 

5.32 

1.86 

3.67 

1.35 

Probability of cumulative graduate GPA > 3.8 



GRE Verbal Reasoning High quartile 

58% 

58% 

71% 

46% 

66% 

46% 

Low quartile 

36% 

30% 

50% 

16% 

31% 

22% 

High/low 

1.62 

1.96 

1.43 

2.95 

2.16 

2.15 

GRE Quantitative Reasoning High quartile 

55% 

61% 

78% 

40% 

59% 

45% 

Low quartile 

38% 

38% 

47% 

16% 

30% 

19% 

High/low 

1.45 

1.60 

1.65 

2.57 

1.96 

2.34 

GRE Analytical Writing High quartile 

58% 

55% 

74% 

42% 

64% 

46% 

Low quartile 

36% 

30% 

52% 

14% 

35% 

16% 

High/low 

1.59 

1.82 

1.43 

3.05 

1.85 

2.84 


Note. CIP = classification of instructional programs. 
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Table A17 GRE Quartile Comparisons: Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

Computer 8t 
information 
sciences (11) 

Education 

(13) 

Engineering 

(14) 

Engineering 
technologies & 
engineering 
related fields 
(15) 

Foreign 
languages, 
literature 8t 
linguistics 
(16) 

Family 8t 
consumer 

sciences/ 
human 
sciences (19) 

k (Total # of universities contributing data) 

8 

10 

7 

2 

6 

2 

N (Total # of students contributing data) 

278 

4,649 

1,481 

141 

238 

87 



Probability of grade of C+ or lower 




GRE Verbal Reasoning 

Low quartile 

32% 

17% 

33% 

62% 

18% 

19% 


High quartile 

19% 

12% 

30% 

46% 

6% 

17% 


Low/high 

1.75 

1.40 

1.11 

1.35 

2.82 

1.12 

GRE Quantitative Reasoning 

Low quartile 

31% 

17% 

36% 

74% 

12% 

15% 


High quartile 

17% 

14% 

25% 

44% 

10% 

9% 


Low/high 

1.75 

1.23 

1.43 

1.67 

1.21 

1.61 

GRE Analytical Writing 

Low quartile 

33% 

18% 

40% 

68% 

17% 

17% 


High quartile 

21% 

11% 

25% 

48% 

8% 

6% 


Low/high 

1.60 

1.57 

1.57 

1.42 

2.24 

2.84 


Probability of cumulative graduate GPA > 3.8 



GRE Verbal Reasoning 

High quartile 

37% 

76% 

37% 

32% 

70% 

62% 


Low quartile 

19% 

54% 

27% 

11% 

48% 

75% 


High/low 

1.94 

1.41 

1.35 

2.97 

1.46 

0.82 

GRE Quantitative Reasoning 

High quartile 

38% 

71% 

37% 

26% 

66% 

63% 


Low quartile 

23% 

55% 

25% 

6% 

52% 

70% 


High/low 

1.68 

1.29 

1.46 

4.46 

1.27 

0.89 

GRE Analytical Writing 

High quartile 

42% 

76% 

39% 

32% 

73% 

76% 


Low quartile 

22% 

55% 

23% 

3% 

49% 

68% 


High/low 

1.90 

1.39 

1.70 

10.54 

1.48 

1.12 


Note. CIP = classification of instructional programs. 


Table A18 GRE Quartile Comparisons: Master’s Seekers 


General program area 
(with its 2-digit CIP code) 

English 
language 8t 
literature/ 
letters (23) 

Liberal arts 8t 
sciences, 
general studies & 
humanities (24) 

Library Biological & 
science biomedical 
(25) sciences (26) 

Mathematics & 
statistics 
(27) 

Multi/ 

interdisciplinary 
studies (30) 

k (Total # of universities contributing data) 

9 

6 

2 

9 

8 

4 

N (Total # of students contributing data) 

552 

118 

867 

445 

230 

75 


Probability of grade of C+ or lower 




GRE Verbal Reasoning Low quartile 

12% 

28% 

20% 

22% 

45% 

31% 

High quartile 

6% 

12% 

15% 

13% 

31% 

20% 

Low/high 

2.18 

2.35 

1.36 

1.70 

1.43 

1.53 

GRE Quantitative Reasoning Low quartile 

10% 

28% 

18% 

30% 

49% 

29% 

High quartile 

4% 

23% 

21% 

8% 

31% 

22% 

Low/high 

2.36 

1.21 

0.87 

3.61 

1.57 

1.30 

GRE Analytical Writing Low quartile 

11% 

34% 

24% 

29% 

47% 

23% 

High quartile 

4% 

29% 

15% 

20% 

38% 

5% 

Low/high 

2.76 

1.17 

1.63 

1.49 

1.22 

4.82 

Probability of cumulative graduate GPA > 3.8 



GRE Verbal Reasoning High quartile 

74% 

70% 

68% 

57% 

21% 

52% 

Low quartile 

49% 

26% 

52% 

31% 

14% 

40% 

High/low 

1.51 

2.71 

1.31 

1.84 

1.53 

1.30 

GRE Quantitative Reasoning High quartile 

77% 

47% 

63% 

55% 

29% 

65% 

Low quartile 

55% 

32% 

58% 

28% 

7% 

32% 

High/low 

1.42 

1.47 

1.10 

1.95 

4.22 

1.99 

GRE Analytical Writing High quartile 

75% 

57% 

73% 

48% 

23% 

68% 

Low quartile 

42% 

30% 

47% 

25% 

18% 

41% 

High/low 

1.79 

1.88 

1.56 

1.89 

1.27 

1.65 


Note. CIP = classification of instructional programs. 
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Table A19 GRE Quartile Comparisons: Master’s Seekers 



Parks, 







recreation, 




Security & 

Public 


leisure, & 

Philosophy & 

Physical 


protective 

administration & 

General program area 

fitness studies 

religious 

sciences 

Psychology 

services 

social service 

(with its 2-digit CIP code) 

(31) 

studies (38) 

(40) 

(42) 

(43) 

professions (44) 

k (Total # of universities contributing data) 

6 

5 

7 

9 

7 

9 

N (Total # of students contributing data) 

375 

81 

240 

461 

469 

1,713 


Probability of grade of C+ or lower 




GRE Verbal Reasoning Low quartile 

39% 

10% 

19% 

12% 

28% 

22% 

High quartile 

29% 

5% 

20% 

6% 

18% 

11% 

Low/high 

1.35 

1.96 

0.95 

2.05 

1.59 

2.04 

GRE Quantitative Reasoning Low quartile 

37% 

11% 

15% 

11% 

27% 

23% 

High quartile 

30% 

7% 

14% 

6% 

14% 

9% 

Low/high 

1.21 

1.49 

1.02 

1.71 

1.84 

2.67 

GRE Analytical Writing Low quartile 

36% 

5% 

25% 

17% 

29% 

23% 

High quartile 

21% 

16% 

18% 

5% 

14% 

10% 

Low/high 

1.76 

0.32 

1.37 

3.47 

2.14 

2.25 

Probability of cumulative graduate GPA > 3.8 




GRE Verbal Reasoning High quartile 

42% 

44% 

34% 

62% 

50% 

61% 

Low quartile 

24% 

26% 

21% 

48% 

20% 

32% 

High/low 

1.76 

1.72 

1.62 

1.28 

2.50 

1.95 

GRE Quantitative Reasoning High quartile 

41% 

58% 

30% 

56% 

42% 

65% 

Low quartile 

30% 

26% 

31% 

54% 

32% 

34% 

High/low 

1.38 

2.19 

0.98 

1.03 

1.32 

1.89 

GRE Analytical Writing High quartile 

44% 

52% 

33% 

59% 

46% 

65% 

Low quartile 

19% 

47% 

19% 

45% 

23% 

33% 

High/low 

2.25 

1.11 

1.72 

1.31 

1.99 

1.95 

Note. CIP = classification of instructional programs. 


Table A20 GRE Quartile Comparisons: Master’ 

s Seekers 







Social 

Visual 8t 

Health 

Business, 


General program area 

sciences 

performing 

professions & 

management, 


(with its 2-digit CIP code) 

(45) 

arts (50) 

clinical sciences (51) 

marketing (52) 

History (54) 

k (Total # of universities contributing data) 

9 

6 

10 

9 

8 

N (Total # of students contributing data) 

1,133 

744 

3,772 

901 

289 



Probability of grade of C+ or lower 



GRE Verbal Reasoning 

Low quartile 

22% 

22% 

30% 

43% 

8% 


High quartile 

12% 

9% 

17% 

31% 

10% 


Low/high 

1.81 

2.37 

1.76 

1.39 

0.76 

GRE Quantitative Reasoning 

Low quartile 

25% 

18% 

28% 

42% 

14% 


High quartile 

14% 

9% 

19% 

33% 

10% 


Low/high 

1.80 

1.88 

1.45 

1.28 

1.51 

GRE Analytical Writing 

Low quartile 

24% 

17% 

30% 

45% 

9% 


High quartile 

14% 

9% 

18% 

33% 

11% 


Low/high 

1.72 

1.79 

1.68 

1.37 

0.83 


Probability of cumulative graduate GPA > 3.8 



GRE Verbal Reasoning 

High quartile 

59% 

59% 

57% 

27% 

62% 


Low quartile 

21% 

47% 

33% 

13% 

28% 


High/low 

2.83 

1.27 

1.71 

2.09 

2.19 

GRE Quantitative Reasoning 

High quartile 

46% 

55% 

55% 

25% 

52% 


Low quartile 

26% 

48% 

36% 

19% 

39% 


High/low 

1.80 

1.15 

1.54 

1.32 

1.34 

GRE Analytical Writing 

High quartile 

48% 

59% 

54% 

29% 

60% 


Low quartile 

27% 

48% 

36% 

17% 

30% 


High/low 

1.82 

1.24 

1.52 

1.73 

1.97 


Note. CIP = classification of instructional programs. 
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Table A21 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for 
All Program Areas: Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Overall 

Agriculture & 
related 
sciences (01) 

Natural 
resources & 
conservation 
(03) 

Architecture 
& related 
services 
(04) 

Communication, 

journalism, 

related 

programs (09) 

Computer 8t 
information 
sciences (11) 

k (Total # of universities co 

ntributing data) 

10 

1 

1 

2 

2 

4 

N (Total # of students contributing data) 

4,229 

46 

26 

13 

23 

102 

Attenuation of p due to 

Cronbach’s alpha 

0.66 

0.68 

0.75 

0.77 

0.36 

0.76 

unreliability of criterion 

Reduction in 

-19% 

-18% 

-13% 

-12% 

-40% 

-13% 

(cumulative graduate 

magnitude of p XY 







GPA) 








GRE Verbal Reasoning 

Mean Ujm 

1.14 

1.04 

1.05 

1.68 

1.81 

1.16 


r obs 

0.13 

-0.04 

-0.05 

0.36 

-0.27 

0.02 


Grv 

0.15 

-0.04 

-0.05 

0.54 

-0.42 

0.03 


r 

0.16 

0.00 

-0.05 

0.86 

-0.44 

0.03 


r mrv+UGPA 

0.19 

0.04 

-0.03 

0.13 

-0.51 

0.05 

GRE Quantitative 

Mean 

1.21 

1.24 

1.09 

1.37 

1.37 

1.48 

Reasoning 

r obs 

0.15 

0.03 

-0.21 

0.04 

0.20 

0.17 


Grv 

0.17 

0.03 

-0.23 

0.08 

0.26 

0.23 


r 

0.20 

0.07 

-0.16 

0.69 

0.06 

0.26 


r mrv+UGPA 

0.24 

0.08 

-0.15 

0.42 

0.01 

0.24 

GRE Analytical Writing 

Mean U B 

1.09 

1.22 

1.12 

0.97 

0.98 

1.16 


r obs 

0.16 

0.16 

0.21 

0.46 

0.07 

0.02 


Cirv 

0.16 

0.19 

0.24 

0.45 

0.05 

0.02 


r 

0.17 

0.20 

0.17 

0.42 

-0.09 

0.00 


r mrv+UGPA 

0.21 

0.22 

0.18 

0.18 

-0.10 

0.11 


Note. CIP = classification of instructional programs. 


Table A22 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for 
All Program Areas: Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Education 

(13) 

Engineering 

(14) 

Foreign 
languages, 
literature, & 
linguistics 
(16) 

Family 8t 
consumer 

sciences/ 
human 
sciences (19) 

English 
language 8t 
literature/ 
letters 
(23) 

k (Total # of universities contributing data) 

8 

6 

3 

1 

4 

N (Total # of students contributing data) 

539 

670 

50 

12 

137 

Attenuation of p due to 

Cronbach’s alpha 

0.65 

0.62 

0.68 

0.57 

0.59 

unreliability of criterion 

Reduction in 

-20% 

-21% 

-17% 

-24% 

-24% 

(cumulative graduate GPA) 
GRE Verbal Reasoning 

magnitude of p XY 
Mean Urh 

1.04 

1.22 

0.95 

0.83 

1.23 


r obs 

0.18 

0.04 

0.10 

0.48 

0.01 


Cirv 

0.18 

0.05 

0.09 

0.41 

0.00 


r 

0.21 

0.04 

0.23 

0.13 

0.03 


r mrv+UGPA 

0.23 

0.08 

0.24 

0.10 

0.09 

GRE Quantitative Reasoning 

Mean Urk 

1.06 

1.10 

1.21 

1.24 

1.04 


fobs 

0.19 

0.14 

0.35 

-0.36 

0.06 


Grv 

0.20 

0.15 

0.37 

-0.44 

0.05 


r 

0.29 

0.21 

0.39 

-0.53 

0.07 


r mrv+UGPA 

0.30 

0.24 

0.32 

-0.53 

0.10 

GRE Analytical Writing 

Mean 

1.03 

1.09 

1.08 

0.90 

1.20 


fobs 

0.15 

0.06 

0.22 

0.04 

0.07 


Grv 

0.15 

0.06 

0.23 

0.04 

0.07 


r 

0.17 

0.07 

0.28 

-0.37 

0.01 


r mrv+UGPA 

0.19 

0.10 

0.28 

-0.35 

0.11 


Note. CIP = classification of instructional programs. 
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Table A23 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for 
All Program Areas: Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Liberal arts & 
sciences, general 
studies & 
humanities (24) 

Library 

science 

(25) 

Biological & 
biomedical 
sciences (26) 

Mathematics & 
statistics (27) 

Multi/ 

interdisciplinary 
studies (30) 

k (Total # of universities contributing data) 

1 

1 

6 

5 

4 

N (Total # of students contributing data) 

20 

9 

446 

122 

56 

Attenuation of p XY due to 

Cronbach’s alpha 

0.31 

0.07 

0.61 

0.57 

0.61 

unreliability of criterion 

Reduction in 

-44% 

-73% 

-22% 

-24% 

-22% 

(cumulative graduate GPA) 
GRE Verbal Reasoning 

magnitude of p XY 
Mean Uhk 

1.00 

0.88 

1.17 

1.09 

1.07 


''obs 

0.59 

0.25 

0.24 

0.03 

0.25 


Girv 

0.59 

0.22 

0.28 

0.02 

0.25 


r 

0.55 

0.27 

0.27 

0.19 

0.15 


r mrv+UGPA 

0.54 

0.28 

0.30 

0.20 

0.14 

GRE Quantitative Reasoning 

Mean 

0.96 

1.16 

1.22 

1.48 

1.30 


r obs 

0.47 

0.17 

0.16 

0.29 

0.05 


Grv 

0.45 

0.20 

0.19 

0.39 

0.07 


r 

0.46 

0.28 

0.20 

0.40 

0.13 


r mrv+UGPA 

0.44 

0.33 

0.22 

0.41 

0.15 

GRE Analytical Writing 

Mean 

0.79 

0.81 

1.12 

1.27 

1.12 


r obs 

0.69 

0.23 

0.23 

0.02 

0.11 


Grv 

0.60 

0.19 

0.26 

0.05 

0.13 


r 

0.64 

0.21 

0.27 

0.15 

0.12 


r mrv+UGPA 

0.65 

0.23 

0.27 

0.18 

0.14 


Note. CIP = classification of instructional programs. 


Table A24 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for 
All Program Areas: Doctorate Seekers 


Parks, 

recreation, Public 

leisure, & Philosophy & Physical Security & administration & 


General program area 
(with its 2-digit CIP code) 

fitness studies 
(31) 

religious 
studies (38) 

sciences 

(40) 

Psychology 

(42) 

protective 
services (43) 

social service 
professions (44) 

k (Total # of universities contributing data) 

1 

3 

7 

6 

1 

5 

N (Total # of students contributing data) 

25 

106 

508 

417 

26 

70 

Attenuation of p XY due to Cronbach’s alpha 

0.68 

0.73 

0.72 

0.65 

0.82 

0.66 

unreliability of criterion Reduction in 

-17% 

-14% 

-15% 

-19% 

-10% 

-18% 

(cumulative graduate magnitude of p XY 







GPA) 







GRE Verbal Reasoning Mean Uj^ 

1.36 

1.37 

1.09 

1.22 

0.90 

0.93 

r obs 

0.33 

0.19 

0.04 

0.17 

0.41 

0.26 

'"urv 

0.43 

0.24 

0.05 

0.21 

0.38 

0.24 

r 

0.69 

0.12 

0.07 

0.24 

0.39 

0.32 

r mrv+UGPA 

0.69 

0.18 

0.07 

0.27 

0.39 

0.32 

GRE Quantitative Mean U RR 

1.14 

1.14 

1.18 

1.46 

1.21 

1.25 

Reasoning r obs 

0.41 

-0.18 

0.18 

0.24 

-0.04 

0.08 

r 

0.45 

-0.23 

0.21 

0.31 

-0.05 

0.06 

r 

0.68 

-0.22 

0.22 

0.32 

0.12 

0.12 

r mrv+UGPA 

0.69 

-0.13 

0.21 

0.39 

0.15 

0.27 

GRE Analytical Writing Mean Ujm 

1.06 

1.17 

1.09 

1.04 

0.87 

1.09 

r obs 

0.17 

0.09 

0.17 

0.24 

0.38 

0.34 

r urv 

0.18 

0.11 

0.19 

0.25 

0.34 

0.35 

r 

0.38 

0.08 

0.18 

0.28 

0.36 

0.27 

r miv+UGPA 

0.37 

0.14 

0.18 

0.32 

0.46 

0.32 


Note. CIP = classification of instructional programs. 
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Table A25 Zero-Order Correlations Between Cumulative Graduate Grade Point Average (GGPA) and GRE Sections Overall and for 
All Program Areas: Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Social 

sciences 

(45) 

Visual & 
performing 
arts (50) 

Health 

professions & 
clinical sciences (51) 

Business, 
management, 
marketing (52) 

History (54) 

k (Total # of universities contributing data) 

5 

3 

8 

4 

3 

N (Total # of students contributing data) 


326 

38 

337 

16 

89 

Attenuation of p XY due to unreliability of 

Cronbach s alpha 

0.77 

0.35 

0.71 

0.45 

0.66 

criterion (cumulative graduate GPA) 

Reduction in 

magnitude of p xY 

-12% 

-41% 

-16% 

-33% 

-19% 

GRE Verbal Reasoning 

Mean 

1.08 

1.39 

1.09 

2.11 

1.11 


^obs 

0.16 

0.40 

0.10 

-0.12 

0.44 


r 

0.17 

0.48 

0.12 

-0.06 

0.49 


r 

0.18 

0.30 

0.12 

-0.20 

0.47 


r mrv+UGPA 

0.21 

0.30 

0.15 

N/A 

0.47 

GRE Quantitative Reasoning 

Mean Urr 

1.36 

1.14 

1.16 

1.47 

1.10 



0.03 

-0.08 

0.18 

-0.36 

0.21 


y 

0.04 

-0.10 

0.20 

-0.42 

0.24 


r 

' mrv 

0.08 

-0.21 

0.22 

-0.55 

0.24 


r mrv+UGPA 

0.10 

-0.18 

0.24 

N/A 

0.22 

GRE Analytical Writing 

Mean Upp 

1.10 

1.12 

1.02 

1.45 

1.14 


*”obs 

0.19 

-0.02 

0.17 

-0.12 

0.20 


r mv 

0.20 

-0.10 

0.17 

-0.16 

0.22 


r 

0.21 

0.16 

0.20 

-0.22 

0.25 


r mrv+UGPA 

0.23 

0.20 

0.23 

N/A 

0.29 


Note. CIP = classification of instructional programs. 


Table A26 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Doctorate 
Seekers 


General program area 
(with its 2-digit CIP code) 

Overall 

Agriculture 
& related 
sciences 
(01) 

Natural 
resources & 
conservation 
(03) 

Architecture Communication, Computer & 
& related journalism, information 

services related sciences 

(04) programs (09) (11) 

Education 

(13) 

k (Total # of universities contributing data) 

10 

1 

1 

2 

2 

4 

8 

N (Total # of students contributing data) 

4,229 

46 

26 

13 

23 

102 

539 

Model R (R 2 ) 

R(R 2 ) 

0.37 (0.13) 

0.20 (0.04) 

0.32 (0.10) 

0.77 (0.59) 

0.52 (0.27) 

0.32 (0.10) 

0.33 (0.11) 


r +ugpa + ugpa ) 

0.46 (0.21) 

0.27 (0.07) 

0.34 (0.11) 

0.78 (0.62) 

0.52 (0.27) 

0.43 (0.18) 

0.37 (0.14) 

AR 2 over UGPA 

GRE V + Q + AW 

0.15 

0.02 

0.11 

0.61 

0.27 

0.17 

0.12 


GRE Verbal 

0.05 

0.00 

0.00 

0.13 

0.13 

0.02 

0.06 


Reasoning 









GRE Quantitative 

0.05 

0.00 

0.05 

0.16 

0.08 

0.06 

0.05 


Reasoning 









GRE Analytical 

0.05 

0.02 

0.05 

0.19 

0.02 

0.07 

0.04 


Writing 








GRE Verbal 

p 

0.06 

-0.14 

0.00 

1.15 

-0.44 

0.03 

0.10 

Reasoning 

P+UGVh 

0.04 

-0.09 

0.00 

0.49 

-0.50 

-0.02 

0.10 


%c xj 

30% 

21% 

3% 

36% 

60% 

27% 

24% 


%( -"xj+UGPA 

17% 

8% 

3% 

25% 

60% 

17% 

21% 

GRE Quantitative 

P 

0.12 

0.03 

-0.24 

0.93 

0.29 

0.23 

0.17 

Reasoning 

P+ UGPA 

0.11 

0.00 

-0.25 

0.81 

0.29 

0.16 

0.17 


%c xj 

34% 

2% 

47% 

21% 

27% 

61% 

40% 


%C x j +UG p A 

21% 

1% 

44% 

21% 

28% 

58% 

24% 

GRE Analytical 

p 

0.11 

0.21 

0.25 

0.20 

0.20 

-0.04 

0.07 

Writing 

P+ UGPA 

0.11 

0.18 

0.25 

0.50 

0.26 

0.07 

0.07 


%c xj 

35% 

77% 

52% 

40% 

13% 

13% 

35% 


% C x j+UGPA 

21% 

35% 

48% 

31% 

12% 

9% 

31% 

UGPA 

P 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


P+vsGm 

0.22 

0.19 

0.10 

-0.42 

-0.07 

0.15 

0.13 


%C xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


%C x j +UG p A 

40% 

59% 

7% 

28% 

4% 

19% 

23% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; AW = Analytical 
Writing. 
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Table A27 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Doctorate 
Seekers 


General program area 
(with its 2-digit CIP code) 

Engineering 

(14) 

Foreign 
languages, 
literature, 8c 
linguistics 
(16) 

Family 8c 
consumer 

sciences/ 

human 

sciences 

(19) 

English 
language 8c 
literature/ 
letters (23) 

Liberal arts 8c 
sciences, 
general 
studies 8c 
humanities 
(24) 

k (Total # of universities contributing data) 

6 

3 

1 

4 

1 

N (Total # of students contributing data) 

670 

50 

12 

137 

20 

Model R (R 2 ) 

R (R 2 ) 

0.26 (0.07) 

0.55 (0.30) 

0.76 (0.57) 

0.37 (0.14) 

0.75 (0.56) 


R+ugpa (^ 2 +ugpa) 

0.44 (0.20) 

0.68 (0.46) 

0.82 (0.67) 

0.48 (0.23) 

0.76 (0.58) 

AR 2 over UGPA 

GRE V + Q + AW 

0.07 

0.36 

0.66 

0.21 

0.51 


GRE Verbal Reasoning 

0.01 

0.03 

0.22 

0.06 

0.30 


GRE Quantitative 
Reasoning 

0.03 

0.16 

0.21 

0.04 

0.18 


GRE Analytical Writing 

0.01 

0.06 

0.00 

0.09 

0.44 

GRE Verbal Reasoning 

P 

-0.03 

-0.14 

0.79 

-0.02 

0.21 


P+VGVK 

-0.01 

-0.07 

0.76 

-0.01 

0.16 


%c xj 

32% 

16% 

49% 

33% 

27% 


^Qjj+UGPA 

6% 

9% 

41% 

17% 

24% 

GRE Quantitative Reasoning 

P 

0.15 

0.35 

-0.61 

0.08 

0.24 


P+ UGPA 

0.09 

0.31 

-0.84 

0.07 

0.24 


%c xj 

38% 

52% 

30% 

19% 

18% 


%C X j +UGPA 

20% 

39% 

32% 

13% 

17% 

GRE Analytical Writing 

p 

0.06 

0.08 

-0.66 

0.05 

0.47 


P+ UGPA 

0.03 

0.07 

-0.76 

0.07 

0.61 


%c xj 

28% 

28% 

14% 

47% 

49% 


%C X j +UGPA 

4% 

8% 

14% 

36% 

47% 

UGPA 

p 

N/A 

N/A 

N/A 

N/A 

N/A 


P+ UGPA 

0.35 

0.23 

0.37 

0.12 

-0.18 


%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 


^Qjj+UGPA 

69% 

40% 

7% 

33% 

5% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A28 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Doctorate 
Seekers 


General program area 
(with its 2-digit CIP code) 

Library 

science 

(25) 

Biological 8c 
biomedical 
sciences 
(26) 

Mathematics 8c 
statistics (27) 

Multi/ 

interdisciplinary 
studies (30) 

Parks, 
recreation, 
leisure, 8c 
fitness studies 
(31) 

k (Total # of universities contributing data) 

1 

6 

5 

4 

1 

N (Total # of students contributing data) 

9 

446 

122 

56 

25 

Model R (R 2 ) R (R 2 ) 

0.30 (0.09) 

0.35 (0.12) 

0.41 (0.17) 

0.42 (0.18) 

0.62 (0.39) 

R+UGPA +ugpa) 

0.34 (0.12) 

0.41 (0.17) 

0.51 (0.26) 

0.51 (0.26) 

0.63 (0.39) 

AR 2 over UGPA GRE V + Q + AW 

0.12 

0.10 

0.23 

0.24 

0.39 

GRE Verbal Reasoning 

0.06 

0.05 

0.04 

0.05 

0.11 

GRE Quantitative Reasoning 

0.03 

0.03 

0.10 

0.03 

0.17 

GRE Analytical Writing 

0.05 

0.05 

0.05 

0.03 

0.03 

GRE Verbal /? 

0.18 

0.16 

0.04 

0.13 

0.48 

Reasoning P+vgpa 

0.16 

0.16 

0.01 

0.05 

0.49 

%c xj 

40% 

36% 

25% 

51% 

38% 

%C X ; +UGPA 

32% 

25% 

15% 

34% 

38% 
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Table A28 Continued 


General program area 
(with its 2-digit CIP code) 

Library 

science 

(25) 

Biological 8c 
biomedical 
sciences 
(26) 

Mathematics 8c 
statistics (27) 

Multi/ 

interdisciplinary 
studies (30) 

Parks, 
recreation, 
leisure, 8c 
fitness studies 
(31) 

GRE Quantitative 

P 

0.17 

0.09 

0.28 

-0.05 

0.55 

Reasoning 

P+OGVK 

0.28 

0.07 

0.25 

-0.04 

0.56 


%c X) 

30% 

20% 

60% 

28% 

58% 


%Cjj +UG p A 

31% 

13% 

39% 

24% 

57% 

GRE Analytical 

p 

0.08 

0.16 

0.00 

0.14 

0.04 

Writing 

P+ UGPA 

0.17 

0.12 

0.01 

0.13 

0.02 


%C X) 

30% 

41% 

14% 

21% 

6% 


^Cxj+UGPA 

26% 

24% 

11% 

12% 

5% 

UGPA 

P 

N/A 

N/A 

N/A 

N/A 

N/A 


P+ UGPA 

-0.21 

0.21 

0.19 

0.14 

0.07 


%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 


%C X j+UGPA 

9% 

35% 

36% 

29% 

0% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A29 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Doctorate 
Seekers 


General program area 
(with its 2-digit CIP code) 

Philosophy 8c 
religious 
studies 
(38) 

Physical 

sciences 

(40) 

Psychology 

(42) 

Security 8c 
protective 
services 
(43) 

Public 

administration 8c 
social service 
professions (44) 

k (Total # of universities contributing data) 

3 

7 

6 

1 

5 

N (Total # of students contributing data) 

106 

508 

417 

26 

70 

Model R (R 2 ) 

R(R 2 ) 

0.44(0.19) 

0.31 (0.09) 

0.37 (0.14) 

0.50 (0.25) 

0.62 (0.38) 


R+UGPA (^~+ugpa) 

0.53 (0.28) 

0.38 (0.15) 

0.53 (0.28) 

0.59 (0.34) 

0.68 (0.46) 

AR 2 over UGPA 

GRE V + Q + AW 

0.15 

0.09 

0.13 

0.27 

0.46 


GRE Verbal Reasoning 

0.07 

0.02 

0.02 

0.14 

0.17 


GRE Quantitative 
Reasoning 

0.06 

0.02 

0.07 

0.00 

0.08 


GRE Analytical Writing 

0.05 

0.03 

0.05 

0.20 

0.18 

GRE Verbal 

P 

0.18 

-0.07 

0.02 

0.36 

0.15 

Reasoning 

P+ UGPA 

0.14 

-0.08 

-0.04 

0.29 

-0.11 


%c XJ 

52% 

19% 

18% 

55% 

36% 


%Cxj+UGPA 

23% 

14% 

6% 

36% 

32% 

GRE Quantitative 

P 

-0.27 

0.18 

0.20 

-0.09 

0.07 

Reasoning 

P+ UGPA 

-0.20 

0.14 

0.22 

-0.10 

0.54 


%C XJ 

28% 

36% 

41% 

3% 

17% 


%C X j+UGPA 

19% 

18% 

22% 

2% 

16% 

GRE Analytical 

P 

0.07 

0.17 

0.19 

0.27 

0.28 

Writing 

P+ UGPA 

0.08 

0.16 

0.17 

0.35 

0.39 


%C xj 

19% 

43% 

38% 

43% 

45% 


%Otj+UGPA 

13% 

29% 

18% 

39% 

41% 

UGPA 

p 

N/A 

N/A 

N/A 

N/A 

N/A 


P- t-UGPA 

0.29 

0.21 

0.36 

0.31 

-0.19 


%C XJ 

N/A 

N/A 

N/A 

N/A 

N/A 


%C X j+UGPA 

46% 

37% 

52% 

25% 

9% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 
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Table A30 Multivariate Analyses of Variables’ Importance Based on Observed Data (No Adjustments for Range Variation): Doctorate 
Seekers 


General program area 
(with its 2-digit CIP code) 


Social Visual & 

sciences performing 

(45) arts (50) 

Health 

professions & 
clinical 
sciences (51) 

Business, 

management, 

marketing 

(52) 

History 

(54) 

k (Total # of universities contributing data) 


5 

3 

8 

4 

3 

N (Total # of students contributing data) 


326 

38 

337 

16 

89 

Model R (R 2 ) R (R 2 ) 


0.29 ( 0 . 08 ) 

0.60 ( 0 . 36 ) 

0.40 ( 0 . 16 ) 

1.00 ( 1 . 00 ) 

0.49 ( 0 . 24 ) 

R+ugpa ( r 2 + ugpa ) 


0.39 ( 0 . 15 ) 

0.63 ( 0 . 40 ) 

0.45 ( 0 . 20 ) 

1.00 ( 1 . 00 ) 

0.55 ( 0 . 30 ) 

AR 2 over UGPA GRE V + Q + AW 


0.08 

0.37 

0.17 

0.86 

0.29 

GRE Verbal Reasoning 


0.03 

0.23 

0.05 

0.27 

0.20 

GRE Quantitative Reasoning 

0.01 

0.03 

0.09 

0.27 

0.07 

GRE Analytical Writing 


0.02 

0.10 

0.04 

0.30 

0.08 

GRE Verbal Reasoning /? 


0.10 

0.46 

- 0.02 

0.38 

0.40 

Z^+UGPA 


0.10 

0.50 

- 0.03 

- 0.65 

0.35 

%C XJ 


28 % 

47 % 

32 % 

32 % 

74 % 

% C x j+UGPA 


15 % 

43 % 

22 % 

24 % 

59 % 

GRE Quantitative Reasoning /? 


- 0.02 

- 0.29 

0.17 

- 1.35 

0.00 

P+VGPA 


- 0.04 

- 0.38 

0.14 

- 0.55 

- 0.04 

%C xj 


15 % 

34 % 

45 % 

32 % 

13 % 

% Cxj+UGPA 


8 % 

30 % 

25 % 

24 % 

12 % 

GRE Analytical Writing /? 


0.14 

0.02 

0.15 

- 0.62 

0.09 

P+V GPA 


0.12 

0.06 

0.14 

0.31 

0.16 

%C xi 


56 % 

26 % 

22 % 

36 % 

12 % 

% Cxj+UGPA 


33 % 

23 % 

16 % 

27 % 

10 % 

UGPA 


N/A 

N/A 

N/A 

N/A 

N/A 

P+VGPA 


0.23 

0.05 

0.18 

0.27 

0.18 

%c xj 


N/A 

N/A 

N/A 

N/A 

N/A 

% C x j+UGPA 


42 % 

10 % 

36 % 

46 % 

18 % 

Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q 

= Quantitative; AW = Analytical 

Writing. 







Table A31 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 

Doctorate Seekers 











Communication, 




Agriculture & Natural 

Architecture 

journalism, 

Computer & 



related 

resources & 

& related 

related 

information 

General program area 


sciences 

conservation services 

programs 

sciences 

(with its 2-digit CIP code) 

Overall (01) 

(03) 

(04) 

(09) 

(id 

k (Total # of universities contributing data) 

10 

i 

1 

2 

2 

4 

N (Total # of students contributing data) 

4,229 

46 

26 

13 

23 

102 

Model R(R 2 ) R(R 2 ) 

0.47 ( 0 . 22 ) 0.23 ( 0 . 05 ) 

0.30 ( 0 . 09 ) 

0.96 ( 0 . 92 ) 

0.69 ( 0 . 48 ) 

0.45 ( 0 . 20 ) 

r +ugpa ( r 2 + ugpa ) 

0.55 ( 0 . 30 ) 0.32 ( 0 . 10 ) 

0.32 ( 0 . 10 ) 

0.79 ( 0 . 62 ) 

0.71 ( 0 . 50 ) 

0.57 ( 0 . 33 ) 

AR 2 over UGPA GRE V + Q + AW 

0.12 

0.02 

0.11 

0.49 

0.27 

0.12 

GRE Verbal Reasoning 

0.05 

0.00 

0.00 

N/A 

0.30 

0.05 

GRE Quantitative Reasoning 

0.07 

0.00 

0.03 

N/A 

0.10 

0.09 

GRE Analytical Writing 

0.05 

0.03 

0.03 

N/A 

0.07 

0.06 

GRE Verbal %C x j 

27 % 

16 % 

5 % 

66 % 

61 % 

24 % 

Reasoning % C x j+UGPA 

16 % 

3 % 

5 % 

N/A 

67 % 

15 % 

GRE Quantitative %C x j 

40 % 

4 % 

43 % 

32 % 

26 % 

68 % 

Reasoning % Cxj+UGPA 

22 % 

1 % 

37 % 

N/A 

17 % 

60 % 

GRE Analytical %Cjg 

33 % 

80 % 

52 % 

2 % 

12 % 

8 % 

Writing % Cxj+UGPA 

20 % 

36 % 

51 % 

N/A 

14 % 

12 % 

UGPA %C xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

% C x j +UGPA 

42 % 

60 % 

8 % 

N/A 

2 % 

13 % 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; AW = Analytical 
Writing. 
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Table A32 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Education 

(13) 

Engineering 

(14) 

Foreign 
languages, 
literature, & 
linguistics 
(16) 

Family 8t 
consumer 

sciences/ 
human 
sciences (19) 

English 
language & 
literature/ 
letters (23) 

k (Total # of universities contributing data) 

8 

6 

3 

1 

4 

N (Total # of students contributing data) 

539 

670 

50 

12 

137 

Model R (R 2 ) 

R(R 2 ) 

0.41 (0.17) 

0.35 (0.12) 

0.50 (0.25) 

0.82 (0.67) 

0.46 (0.21) 


R+UGPA (R +ugpa) 

0.47 (0.22) 

0.51 (0.26) 

0.74 (0.54) 

0.90 (0.81) 

0.58 (0.33) 

AR 2 over UGPA 

GRE V + Q + AW 

0.10 

0.06 

0.22 

0.66 

0.13 


GRE Verbal Reasoning 

0.06 

0.01 

0.03 

0.00 

0.08 


GRE Quantitative Reasoning 

0.07 

0.05 

0.21 

0.43 

0.04 


GRE Analytical Writing 

0.04 

0.02 

0.05 

0.15 

0.10 

GRE Verbal Reasoning 

%c xj 

24% 

31% 

12% 

23% 

36% 


% < -' X j +UGPA 

21% 

6% 

7% 

12% 

14% 

GRE Quantitative Reasoning 

%c xj 

43% 

41% 

54% 

51% 

17% 


^Cxj+UGPA 

24% 

22% 

44% 

47% 

10% 

GRE Analytical Writing 

%c xj 

33% 

28% 

34% 

26% 

48% 


% < -' X j +UGPA 

31% 

5% 

5% 

21% 

32% 

UGPA 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 


%C x j +UGPA 

24% 

67% 

44% 

20% 

44% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A33 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Liberal arts & 
sciences, 
general 
studies & 
humanities 
(24) 

Library 

science 

(25) 

Biological & 
biomedical 
sciences 
(26) 

Mathematics 8t 
statistics 
(27) 

Multi/ 

interdisciplinary 
studies (30) 

k (Total # of universities contributing data) 

1 

1 

6 

5 

4 

N (Total # of students contributing data) 

20 

9 

446 

122 

56 

Model R (R 2 ) 

R(R 2 ) 

0.71 (0.50) 

0.33 (0.11) 

0.40 (0.16) 

0.52 (0.27) 

0.52 (0.27) 


R+UGPA +ugpa) 

0.73 (0.53) 

0.45 (0.21) 

0.47 (0.22) 

0.59 (0.34) 

0.59 (0.35) 

AR 2 over UGPA 

GRE V + Q + AW 

0.51 

0.12 

0.10 

0.15 

0.13 


GRE Verbal 
Reasoning 

0.27 

0.10 

0.06 

0.04 

0.03 


GRE Quantitative 
Reasoning 

0.18 

0.15 

0.04 

0.14 

0.08 


GRE Analytical 
Writing 

0.43 

0.08 

0.06 

0.03 

0.05 

GRE Verbal Reasoning 

%c xj 

26% 

38% 

36% 

18% 

47% 


%Ocj+UGPA 

24% 

20% 

26% 

11% 

26% 

GRE Quantitative Reasoning 

%C xj 

21% 

50% 

21% 

73% 

30% 


%C X j+UGPA 

16% 

39% 

13% 

45% 

28% 

GRE Analytical Writing 

%C xj 

53% 

11% 

43% 

10% 

23% 


%C X j+UGPA 

57% 

13% 

26% 

9% 

17% 

UGPA 

%C xj 

N/A 

N/A 

N/A 

N/A 

N/A 


%Ocj+UGPA 

3% 

27% 

35% 

36% 

30% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 
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Table A34 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Parks, 
recreation, 
leisure, & 
fitness 
studies (31) 

Philosophy & 
religious 
studies 
(38) 

Physical 

sciences 

(40) 

Psychology 

(42) 

Security & 
protective 
services 
(43) 

k (Total # of universities contributing data) 

1 

3 

7 

6 

1 

N (Total # of students contributing data) 

25 

106 

508 

417 

26 

Model R (R 2 ) 

R(R 2 ) 

0.80 (0.64) 

0.46 (0.21) 

0.32 (0.11) 

0.43 (0.18) 

0.45 (0.20) 


R+UGPA (^~+ugpa) 

0.81 (0.66) 

0.55 (0.30) 

0.42 (0.17) 

0.63 (0.39) 

0.60 (0.36) 

AR 2 over UGPA 

GRE V + Q + AW 

0.39 

0.14 

0.08 

0.11 

0.27 


GRE Verbal Reasoning 

0.45 

0.08 

0.02 

0.03 

0.11 


GRE Quantitative Reasoning 

0.44 

0.06 

0.03 

0.11 

0.01 


GRE Analytical Writing 

0.12 

0.05 

0.03 

0.05 

0.14 

GRE Verbal Reasoning 

%c xj 

47% 

35% 

16% 

16% 

57% 


^Otj+UGPA 

49% 

27% 

12% 

5% 

23% 

GRE Quantitative Reasoning 

%c xj 

50% 

37% 

45% 

55% 

3% 


^Otj+UGPA 

44% 

17% 

19% 

23% 

1% 

GRE Analytical Writing 

%c xj 

3% 

28% 

39% 

30% 

40% 


^Otj+UGPA 

5% 

13% 

26% 

12% 

35% 

UGPA 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 


^Otj+UGPA 

1% 

43% 

43% 

59% 

40% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 


Table A35 Multivariate Analyses of Variables’ Importance Based on Observed Data (Adjusted for Multivariate Range Variation): 
Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Public 

administration & 
social service 
professions (44) 

Social 

sciences 

(45) 

Elealth 

Visual & professions & 
performing clinical sciences 
arts (50) (51) 

Business, 
management, 
marketing (52) 

Elistory 

(54) 

k (Total # of universities contributing data) 

5 

5 

3 

8 

4 

3 

N (Total # of students contributing data) 

70 

326 

38 

337 

16 

89 

Model R (R 2 ) 

R (R 2 ) 

0.65 (0.42) 

0.36 (0.13) 

0.61 (0.38) 

0.43 (0.18) 

1.00(1.00) 

0.53 (0.28) 


R+UGPA (^ +ugpa) 

0.73 (0.53) 

0.47 (0.22) 

0.63 (0.39) 

0.53 (0.28) 

N/A (N/A) 

0.61 (0.37) 

AR 2 over UGPA 

GRE V + Q + AW 

0.43 

0.05 

0.36 

0.16 

0.47 

0.26 


GRE Verbal 

Reasoning 

0.06 

0.03 

0.23 

0.06 

N/A 

0.21 


GRE Quantitative 
Reasoning 

0.12 

0.02 

0.10 

0.12 

N/A 

0.07 


GRE Analytical 
Writing 

0.14 

0.03 

0.03 

0.07 

N/A 

0.09 

GRE Verbal 

%c xj 

32% 

26% 

49% 

18% 

N/A 

74% 

Reasoning 

%Cxj+UGPA 

22% 

15% 

35% 

13% 

N/A 

58% 

GRE Quantitative 

%c xj 

30% 

21% 

41% 

60% 

N/A 

12% 

Reasoning 

%Cxj+UGPA 

30% 

9% 

36% 

36% 

N/A 

9% 

GRE Analytical 

%c xj 

38% 

53% 

10% 

21% 

N/A 

14% 

Writing 

%C x j+UGPA 

40% 

31% 

10% 

18% 

N/A 

13% 

UGPA 

%c xj 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


%C X j +UGPA 

8% 

45% 

20% 

33% 

N/A 

20% 


Note. CIP = classification of instructional programs; UGPA = undergraduate grade point average; V = Verbal; Q = Quantitative; 
AW = Analytical Writing. 
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Table A36 GRE Quartile Comparisons: Doctorate Seekers 


General program area 

(with its 2-digit CIP code) Overall 

Agriculture 8t 
related 
sciences 
(01) 

Natural 
resources 8t 
conservation 
(03) 

Architecture & 
related services 
(04) 

Communication, 
journalism, 
related programs 
(09) 

Computer 8t 
information 
sciences (11) 

k (Total # of universities contributing data) 

10 

1 

1 

2 

2 

4 

N (Total # of students contributing data) 

4,229 

46 

26 

13 

23 

102 



Probability of grade of C+ or lower 




GRE Verbal Reasoning 

Low quartile 

22% 

36% 

0% 

0% 

0% 

21% 


High quartile 

14% 

25% 

0% 

27% 

20% 

5% 


Low/high 

1.49 

1.45 

N/A 

0.00 

0.00 

4.23 

GRE Quantitative Reasoning 

Low quartile 

21% 

45% 

0% 

0% 

0% 

17% 


High quartile 

15% 

18% 

14% 

0% 

0% 

13% 


Low/high 

1.47 

2.50 

0.00 

N/A 

N/A 

1.29 

GRE Analytical Writing 

Low quartile 

22% 

67% 

20% 

0% 

0% 

17% 


High quartile 

13% 

33% 

0% 

0% 

20% 

6% 


Low/high 

1.64 

2.00 

N/A 

N/A 

0.00 

2.79 


Probability of cumulative graduate GPA > 3.8 



GRE Verbal Reasoning 

High quartile 

62% 

42% 

86% 

73% 

67% 

61% 


Low quartile 

43% 

36% 

100% 

0% 

64% 

54% 


High/low 

1.45 

1.15 

0.86 

N/A 

1.05 

1.13 

GRE Quantitative Reasoning 

High quartile 

60% 

64% 

71% 

73% 

82% 

60% 


Low quartile 

43% 

27% 

100% 

54% 

64% 

34% 


High/low 

1.37 

2.33 

0.71 

1.36 

1.29 

1.75 

GRE Analytical Writing 

High quartile 

60% 

44% 

100% 

100% 

79% 

56% 


Low quartile 

44% 

33% 

80% 

0% 

64% 

59% 


High/low 

1.35 

1.33 

1.25 

N/A 

1.24 

0.95 

Note. CIP = classification of instructional programs. 






Table A37 GRE Quartile Comparisons: Doctorate Seekers 










Foreign 

Family & 







languages, 

consumer 

English 






literature, 8t 

sciences/ 

language 8t 

General program area 

Education Engineering 

linguistics 

human sciences 

literature/ 

(with its 2-digit CIP code) 


(13) 

(14) 

(16) 

(19) 

letters (23) 

k (Total # of universities contributing data) 


8 

6 

3 

1 

4 

N (Total # of students contributing data) 


539 

670 

50 

12 

137 



Probability of grade of C+ or lower 




GRE Verbal Reasoning 

Low quartile 


16% 

34% 

10% 

0% 

0% 


High quartile 


8% 

25% 

6% 

0% 

6% 


Low/high 


1.90 

1.40 

1.63 

N/A 

0.00 

GRE Quantitative Reasoning 

Low quartile 


22% 

25% 

9% 

33% 

0% 


High quartile 


9% 

23% 

0% 

0% 

3% 


Low/high 


2.40 

1.09 

N/A 

N/A 

0.00 

GRE Analytical Writing 

Low quartile 


13% 

32% 

7% 

0% 

0% 


High quartile 


11% 

22% 

6% 

0% 

5% 


Low/high 


1.19 

1.45 

1.22 

N/A 

0.00 


Probability of cumulative graduate GPA > 3.8 



GRE Verbal Reasoning 

High quartile 


83% 

46% 

85% 

50% 

94% 


Low quartile 


58% 

32% 

50% 

33% 

88% 


High/low 


1.43 

1.42 

1.70 

1.50 

1.07 

GRE Quantitative Reasoning 

High quartile 


82% 

51% 

86% 

33% 

88% 


Low quartile 


53% 

35% 

55% 

50% 

83% 


High/low 


1.54 

1.46 

1.57 

0.67 

1.06 

GRE Analytical Writing 

High quartile 


77% 

49% 

85% 

0% 

92% 


Low quartile 


63% 

41% 

70% 

50% 

89% 


High/low 


1.22 

1.21 

1.23 

0.00 

1.04 


Note. CIP = classification of instructional programs. 
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Table A38 GRE Quartile Comparisons: Doctorate Seekers 




Liberal arts & sciences, 

Library 

Biological & 


Multi/ 

General program area 

general studies & 

science 

biomedical 

Mathematics & 

interdisciplinary 

(with its 2-digit CIP code) 

humanities (24) 

(25) 

sciences (26) 

statistics (27) 

studies (30) 

k (Total # of universities contributing data) 

1 

1 

6 

5 

4 

N (Total # of students contributing data) 

20 

9 

446 

122 

56 



Probability of grade of C+ or lower 



GRE Verbal Reasoning 

Low quartile 

20% 

0% 

27% 

55% 

8% 


High quartile 

0% 

0% 

10% 

25% 

22% 


Low/high 

N/A 

N/A 

2.80 

2.21 

0.38 

GRE Quantitative Reasoning 

Low quartile 

40% 

0% 

23% 

56% 

0% 


High quartile 

0% 

0% 

12% 

29% 

14% 


Low/high 

N/A 

N/A 

1.95 

1.93 

0.00 

GRE Analytical Writing 

Low quartile 

50% 

0% 

21% 

38% 

16% 


High quartile 

0% 

0% 

8% 

44% 

11% 


Low/high 

N/A 

N/A 

2.48 

0.86 

1.47 


Probability of cumulative graduate GPA > 3.8 



GRE Verbal Reasoning 

High quartile 

100% 

100% 

56% 

35% 

55% 


Low quartile 

40% 

100% 

39% 

24% 

31% 


High/low 

2.50 

1.00 

1.43 

1.43 

1.78 

GRE Quantitative Reasoning 

High quartile 

100% 

50% 

55% 

36% 

46% 


Low quartile 

60% 

100% 

39% 

10% 

52% 


High/low 

1.67 

0.50 

1.39 

3.42 

0.88 

GRE Analytical Writing 

High quartile 

100% 

100% 

58% 

20% 

47% 


Low quartile 

25% 

50% 

36% 

25% 

44% 


High/low 

4.00 

2.00 

1.62 

0.79 

1.07 

Note. CIP = classification of instructional programs. 


Table A39 GRE Quartile Comparisons: Doctorate Seekers 

General program area 
(with its 2-digit CIP code) 

Parks, 
recreation, 
leisure, & 
fitness 
studies (31) 

Philosophy & 
religious 
studies (38) 

Physical 

sciences 

(40) 

Psychology 

(42) 

Security & 
protective 
services (43) 

k (Total # of universities contributing data) 

1 

3 

7 

6 

1 

N (Total # of students contributing data) 

25 

106 

508 

417 

26 



Probability of grade of C+ or lower 




GRE Verbal Reasoning 

Low quartile 

33% 

17% 

23% 

10% 

33% 


High quartile 

14% 

4% 

25% 

8% 

0% 


Low/high 

2.33 

3.75 

0.90 

1.34 

N/A 

GRE Quantitative Reasoning 

Low quartile 

40% 

12% 

33% 

11% 

33% 


High quartile 

17% 

16% 

24% 

6% 

0% 


Low/high 

2.40 

0.74 

1.40 

1.95 

N/A 

GRE Analytical Writing 

Low quartile 

25% 

11% 

34% 

18% 

40% 


High quartile 

0% 

8% 

17% 

9% 

29% 


Low/high 

N/A 

1.38 

2.04 

1.94 

1.40 


Probability of cumulative graduate GPA > 3.8 




GRE Verbal Reasoning 

High quartile 

86% 

71% 

38% 

76% 

50% 


Low quartile 

67% 

34% 

26% 

56% 

0% 


High/low 

1.29 

2.06 

1.48 

1.36 

N/A 

GRE Quantitative Reasoning 

High quartile 

83% 

34% 

40% 

75% 

50% 


Low quartile 

20% 

59% 

16% 

64% 

33% 


High/low 

4.17 

0.58 

2.57 

1.17 

1.50 

GRE Analytical Writing 

High quartile 

100% 

55% 

43% 

76% 

14% 


Low quartile 

63% 

44% 

22% 

52% 

0% 


High/low 

1.60 

1.23 

1.96 

1.45 

N/A 


Note. CIP = classification of instructional programs. 
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Table A40 GRE Quartile Comparisons: Doctorate Seekers 


General program area 
(with its 2-digit CIP code) 

Public 

administration & 
social service 
professions (44) 

Social 

sciences 

(45) 

Visual & 
performing 
arts (50) 

Health 

professions 8t 
clinical sciences 
(51) 

Business, 

management, 

marketing 

(52) 

History 

(54) 

k (Total # of universities contributing data) 

5 

5 

3 

8 

4 

3 

N (Total # of students contributing data) 

70 

326 

38 

337 

16 

89 



Probability of grade of C+ or lower 




GRE Verbal Reasoning 

Low quartile 

22% 

17% 

11% 

19% 

25% 

25% 


High quartile 

0% 

11% 

0% 

19% 

33% 

9% 


Low/high 

N/A 

1.50 

N/A 

1.03 

0.75 

2.72 

GRE Quantitative Reasoning 

Low quartile 

19% 

15% 

0% 

22% 

0% 

9% 


High quartile 

13% 

13% 

11% 

14% 

33% 

13% 


Low/high 

1.51 

1.16 

0.00 

1.53 

0.00 

0.72 

GRE Analytical Writing 

Low quartile 

25% 

19% 

0% 

20% 

33% 

5% 


High quartile 

0% 

11% 

0% 

13% 

33% 

6% 


Low/high 

N/A 

1.72 

N/A 

1.59 

1.00 

0.86 


Probability of cumulative graduate GPA > 3.8 




GRE Verbal Reasoning 

High quartile 

80% 

66% 

100% 

58% 

67% 

79% 


Low quartile 

28% 

42% 

45% 

41% 

50% 

43% 


High/low 

2.88 

1.59 

2.22 

1.41 

1.33 

1.83 

GRE Quantitative Reasoning 

High quartile 

60% 

45% 

78% 

63% 

33% 

73% 


Low quartile 

46% 

51% 

79% 

37% 

25% 

53% 


High/low 

1.32 

0.88 

0.99 

1.71 

1.33 

1.38 

GRE Analytical Writing 

High quartile 

87% 

64% 

71% 

47% 

33% 

69% 


Low quartile 

32% 

46% 

65% 

35% 

33% 

56% 


High/low 

2.67 

1.39 

1.09 

1.34 

1.00 

1.23 


Note. CIP = classification of instructional programs. 
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